Whenever a pre-defined amount of data is written to a log file (10 MB by default), JE stops using the current log file and starts writing to a new file. You can change the maximum amount of data contained in each log file by using the DB_ENV->set_lg_max() method. Note that this method can be used at any time during an application's lifetime.

Setting the log file size to something larger than its default value is largely a matter of convenience and a reflection of the application's preference in backup media and frequency. However, if you set the log file size too low relative to your application's traffic patterns, you can cause yourself trouble.

From a performance perspective, setting the log file size to a low value can cause your active transactions to pause their writing activities more frequently than would occur with larger log file sizes. Whenever a transaction completes the log buffer is flushed to disk. Normally other transactions can continue to write to the log buffer while this flush is in progress. However, when one log file is being closed and another created, all transactions must cease writing to the log buffer until the switch over is completed.

Beyond performance concerns, using smaller log files can cause you to use more physical files on disk. As a result, your application could run out of log sequence numbers, depending on how busy your application is.

Every log file is identified with a 10 digit number. Moreover, the maximum number of log files that your application is allowed to create in its lifetime is 2,000,000,000.

For example, if your application performs 6,000 transactions per second for 24 hours a day, and you are logging 500 bytes of data per transaction into 10 MB log files, then you will run out of log files in around 221 years:

     (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 

However, if you were writing 2000 bytes of data per transaction, and using 1 MB log files, then the same formula shows you running out of log files in 5 years time.

All of these time frames are quite long, to be sure, but if you do run out of log files after, say, 5 years of continuous operations, then you must reset your log sequence numbers. To do so:

The logging subsystem's default region size is 60 KB. The logging's region is used to store filenames, and so you may need to increase its size if a large number of files (that is, if you have a very large number of databases) will be opened and registered with JE's log manager.

You can set the size of your logging region by using the DB_ENV->set_lg_regionmax() method. Note that this method can only be called before the first environment handle for your application is opened.

It is possible to configure your logging subsystem such that logs are maintained entirely in memory. When you do this, you give up your transactional durability guarantee. Without log files, you have no way to run recovery so any system or software failures that you might experience can corrupt your databases.

However, by giving up your durability guarantees, you can greatly improve your application's throughput by avoiding the disk I/O necessary to write logging information to disk. In this case, you still retain your transactional atomicity, consistency, and isolation guarantees.

To configure your logging subsystem to maintain your logs entirely in-memory:

For example:

#include <stdio.h>
#include <stdlib.h>

#include "db.h"

int
main(void)
{
    int ret, ret_c;
    u_int32_t db_flags, env_flags;
    DB_ENV *envp;
    const char *db_home_dir = "/tmp/myEnvironment";

    envp = NULL;

    /* Create the environment */
    ret = db_env_create(&envp, 0);
    if (ret != 0) {
        fprintf(stderr, "Error creating environment handle: %s\n",
            db_strerror(ret));
        return (EXIT_FAILURE);
    }

    /* 
     * Indicate that logging is to be performed only in memory. 
     * Doing this means that we give up our transactional durability
     * guarantee.
     */
    envp->set_flags(envp, DB_LOG_INMEMORY, 1);

    /* 
     * Configure the size of our log memory buffer. This must be
     * large enough to hold all the logging information likely
     * to be created for our longest running transaction. The
     * default size for the logging buffer is 1 MB when logging
     * is performed in-memory. For this example, we arbitrarily
     * set the logging buffer to 5 MB.
     */
     ret = envp->set_lg_bsize(envp, 5 * 1024 * 1024);
     if (ret != 0) {
        fprintf(stderr, "Error setting log buffer size: %s\n",
            db_strerror(ret));
        goto err;
     }

    /* 
     * Set the normal flags for a transactional subsystem. Note that
     * we DO NOT specify DB_RECOVER. Also, remember that the logging 
     * subsystem is automatically enabled when we initialize the
     * transactional subsystem, so we do not explicitly enable
     * logging here.
     */
    env_flags = DB_CREATE     |  /* If the environment does not
                                  * exist, create it. */
                DB_INIT_LOCK  |  /* Initialize locking */
                DB_INIT_LOG   |  /* Initialize logging */
                DB_INIT_MPOOL |  /* Initialize the cache */
                DB_THREAD     |  /* Free-thread the env handle. */
                DB_INIT_TXN;     /* Initialize transactions */

    /* Open the environment as normal */
    ret = envp->open(envp, db_home_dir, env_flags, 0);
    if (ret != 0) {
        fprintf(stderr, "Error opening environment: %s\n",
            db_strerror(ret));
        goto err;
    }


     /*
      * From here, you create transactions and perform database operations
      * exactly as you would if you were logging to disk. This part is
      * omitted for brevity.
      */

      ...

err:
    /* Close the databases (omitted) */

    ...

    /* Close the environment */
    if (envp != NULL) {
        ret_c = envp->close(envp, 0);
        if (ret_c != 0) {
            fprintf(stderr, "environment close failed: %s\n",
                db_strerror(ret_c));
            ret = ret_c;
        }
    }

    return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE);
} 

When your application is configured for on-disk logging (the default behavior for transactional applications), log information is stored in-memory until the storage space fills up, or a transaction commit forces the log information to be flushed to disk.

It is possible to increase the amount of memory available to your file log buffer. Doing so improves throughput for long-running transactions, or for transactions that produce a large amount of data.

When you have your logging subsystem configured to maintain your log entirely in memory (see Configuring In-Memory Logging), it is very important to configure your log buffer size because the log buffer must be capable of holding all log information that can accumulate during the longest running transaction. You must make sure that the in-memory log buffer size is large enough that no transaction will ever span the entire buffer. You must also avoid a state where the in-memory buffer is full and no space can be freed because a transaction that started the first log "file" is still active.

When your logging subsystem is configured for on-disk logging, the default log buffer space is 32 KB. When in-memory logging is configured, the default log buffer space is 1 MB.

You can increase your log buffer space using the DB_ENV->set_lg_bsize() method. Note that this method can only be called before the first environment handle for your application is opened.