Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 18 additions & 25 deletions docs/core/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -1075,26 +1075,24 @@ inheriting the ``ILogFormatter`` class and implementing the ``object FormatLogEn

### Buffering logs

Log buffering enables you to buffer logs for a specific request or invocation. Enable log buffering by passing `LogBufferingOptions` when configuring a Logger instance. You can buffer logs at the `Warning`, `Information`, `Debug` or `Trace` level, and flush them automatically on error or manually as needed.
Log buffering enables you to buffer logs for a specific request or invocation. Enable log buffering by setting `LogBuffering.Enabled = true` when configuring a Logger instance. You can buffer logs at the `Warning`, `Information`, `Debug` or `Trace` level, and flush them automatically on error or manually as needed.

!!! tip "This is useful when you want to reduce the number of log messages emitted while still having detailed logs when needed, such as when troubleshooting issues."

=== "LogBufferingOptions"

```csharp hl_lines="5-14"
```csharp hl_lines="5-12"
public class Function
{
public Function()
{
Logger.Configure(logger =>
{
logger.Service = "MyServiceName";
logger.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
MaxBytes = 20480, // Default is 20KB (20480 bytes)
FlushOnErrorLog = true // default true
};
logger.LogBuffering.Enabled = true;
logger.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
logger.LogBuffering.MaxBytes = 20480; // Default is 20KB (20480 bytes)
logger.LogBuffering.FlushOnErrorLog = true; // default true
});

Logger.LogDebug('This is a debug message'); // This is NOT buffered
Expand All @@ -1117,28 +1115,27 @@ Log buffering enables you to buffer logs for a specific request or invocation. E

#### Configuring the buffer

When configuring the buffer, you can set the following options to fine-tune how logs are captured, stored, and emitted. You can configure the following options in the `logBufferOptions` constructor parameter:
When configuring the buffer, you can set the following options to fine-tune how logs are captured, stored, and emitted. You can configure the following options in the `LogBuffering` property:

| Parameter | Description | Configuration | Default |
|---------------------|------------------------------------------------- |--------------------------------------------|---------|
| `Enabled` | Enable or disable log buffering | `True`, `False` | `False` |
| `MaxBytes` | Maximum size of the log buffer in bytes | `number` | `20480` |
| `BufferAtLogLevel` | Minimum log level to buffer | `Trace`, `Debug`, `Information`, `Warning` | `Debug` |
| `FlushOnErrorLog` | Automatically flush buffer when logging an error | `True`, `False` | `True` |

=== "BufferAtLogLevel"

```csharp hl_lines="10"
```csharp hl_lines="10 11"
public class Function
{
public Function()
{
Logger.Configure(logger =>
{
logger.Service = "MyServiceName";
logger.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Warning
};
logger.LogBuffering.Enabled = true;
logger.LogBuffering.BufferAtLogLevel = LogLevel.Warning;
});
}

Expand All @@ -1161,18 +1158,16 @@ When configuring the buffer, you can set the following options to fine-tune how

=== "FlushOnErrorLog"

```csharp hl_lines="10"
```csharp hl_lines="10 11"
public class Function
{
public Function()
{
Logger.Configure(logger =>
{
logger.Service = "MyServiceName";
logger.LogBuffering = new LogBufferingOptions
{
FlushOnErrorLog = false
};
logger.LogBuffering.Enabled = true;
logger.LogBuffering.FlushOnErrorLog = false;
});
}

Expand Down Expand Up @@ -1214,18 +1209,16 @@ When using the `Logger` decorator, you can configure the logger to automatically

=== "FlushBufferOnUncaughtError"

```csharp hl_lines="15"
```csharp hl_lines="14"
public class Function
{
public Function()
{
Logger.Configure(logger =>
{
logger.Service = "MyServiceName";
logger.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug
};
logger.LogBuffering.Enabled = true;
logger.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
});
}

Expand Down Expand Up @@ -1328,7 +1321,7 @@ sequenceDiagram
No, we never buffer logs during cold starts. This is because we want to ensure that logs emitted during this phase are always available for debugging and monitoring purposes. The buffer is only used during the execution of the Lambda function.

3. **How can I prevent log buffering from consuming excessive memory?**
You can limit the size of the buffer by setting the `MaxBytes` option in the `LogBufferingOptions` constructor parameter. This will ensure that the buffer does not grow indefinitely and consume excessive memory.
You can limit the size of the buffer by setting the `MaxBytes` option in the `LogBuffering` property. This will ensure that the buffer does not grow indefinitely and consume excessive memory.

4. **What happens if the log buffer reaches its maximum size?**
Older logs are removed from the buffer to make room for new logs. This means that if the buffer is full, you may lose some logs if they are not flushed before the buffer reaches its maximum size. When this happens, we emit a warning when flushing the buffer to indicate that some logs have been dropped.
Expand Down
8 changes: 3 additions & 5 deletions docs/getting-started/logger/aspnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -392,11 +392,9 @@ var logger = LoggerFactory.Create(builder =>
builder.AddPowertoolsLogger(config =>
{
config.Service = "powertools-aspnet-demo";
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = true
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = true;
});
}).CreatePowertoolsLogger();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ private void InitializeLogger(LoggingAttribute trigger)

// Set operational flags based on current configuration
_isDebug = _currentConfig.MinimumLogLevel <= LogLevel.Debug;
_bufferingEnabled = _currentConfig.LogBuffering != null;
_bufferingEnabled = _currentConfig.LogBuffering?.Enabled == true;
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,10 @@ public bool IsEnabled(LogLevel logLevel)
private bool IsEnabledForConfig(LogLevel logLevel, PowertoolsLoggerConfiguration config)
{
//if Buffering is enabled and the log level is below the buffer threshold, skip logging only if bellow error
if (logLevel <= config.LogBuffering?.BufferAtLogLevel
&& config.LogBuffering?.BufferAtLogLevel != LogLevel.Error
&& config.LogBuffering?.BufferAtLogLevel != LogLevel.Critical)
if (config.LogBuffering?.Enabled == true
&& logLevel <= config.LogBuffering.BufferAtLogLevel
&& config.LogBuffering.BufferAtLogLevel != LogLevel.Error
&& config.LogBuffering.BufferAtLogLevel != LogLevel.Critical)
{
return false;
}
Expand Down
17 changes: 17 additions & 0 deletions libraries/src/AWS.Lambda.Powertools.Logging/LogBufferingOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,23 @@ namespace AWS.Lambda.Powertools.Logging;
/// </summary>
public class LogBufferingOptions
{
/// <summary>
/// Gets or sets whether log buffering is enabled.
/// <para></para>
/// Default is false. Set to true to enable log buffering.
/// </summary>
/// <example>
/// <code>
/// // Enable buffering with default options
/// options.LogBuffering.Enabled = true;
///
/// // Enable buffering with custom options
/// options.LogBuffering.Enabled = true;
/// options.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
/// </code>
/// </example>
public bool Enabled { get; set; }

/// <summary>
/// Gets or sets the maximum size of the buffer in bytes
/// <para></para>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ public PowertoolsLoggerBuilder WithFormatter(ILogFormatter formatter)
/// <returns>The builder instance for method chaining.</returns>
public PowertoolsLoggerBuilder WithLogBuffering(Action<LogBufferingOptions> configure)
{
_configuration.LogBuffering = new LogBufferingOptions();
_configuration.LogBuffering.Enabled = true;
configure?.Invoke(_configuration.LogBuffering);
return this;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,9 @@ namespace AWS.Lambda.Powertools.Logging;
/// <code>
/// builder.Logging.AddPowertoolsLogger(options =>
/// {
/// options.LogBuffering = new LogBufferingOptions
/// {
/// Enabled = true,
/// BufferAtLogLevel = LogLevel.Debug,
/// FlushOnErrorLog = true
/// };
/// options.LogBuffering.Enabled = true;
/// options.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
/// options.LogBuffering.FlushOnErrorLog = true;
/// });
/// </code>
///
Expand Down Expand Up @@ -229,32 +226,35 @@ public JsonSerializerOptions JsonOptions
}
}

private LogBufferingOptions _logBuffering = new LogBufferingOptions();

/// <summary>
/// Enables or disables log buffering. Logs below the specified level will be buffered
/// until the buffer is flushed or an error occurs.
/// Buffer logs at the WARNING, INFO, and DEBUG levels and reduce CloudWatch costs by decreasing the number of emitted log messages
/// Gets or sets the configuration options for log buffering.
/// When enabled, logs below the specified level are buffered until explicitly flushed or an error occurs.
/// This can reduce CloudWatch costs by decreasing the number of emitted log messages.
/// <para>Log buffering is disabled by default. Set <see cref="LogBufferingOptions.Enabled"/> to true to enable it.</para>
/// <para>Setting this property to null will reset it to a default <see cref="LogBufferingOptions"/> instance with buffering disabled.</para>
/// </summary>
/// <example>
/// <code>
/// // Enable buffering for debug logs
/// options.LogBuffering = new LogBufferingOptions
/// {
/// Enabled = true,
/// BufferAtLogLevel = LogLevel.Debug,
/// FlushOnErrorLog = true
/// };
/// // Enable buffering with default options
/// options.LogBuffering.Enabled = true;
///
/// // Enable buffering for debug logs with custom settings
/// options.LogBuffering.Enabled = true;
/// options.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
/// options.LogBuffering.FlushOnErrorLog = true;
///
/// // Buffer all logs below Error level
/// options.LogBuffering = new LogBufferingOptions
/// {
/// Enabled = true,
/// BufferAtLogLevel = LogLevel.Warning,
/// FlushOnErrorLog = true
/// };
/// options.LogBuffering.Enabled = true;
/// options.LogBuffering.BufferAtLogLevel = LogLevel.Warning;
/// </code>
/// </example>
public LogBufferingOptions LogBuffering { get; set; }

public LogBufferingOptions LogBuffering
{
get => _logBuffering;
set => _logBuffering = value ?? new LogBufferingOptions();
}
/// <summary>
/// Serializer instance for this configuration
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,8 @@ public static ILoggingBuilder AddPowertoolsLogger(
/// <code>
/// builder.Logging.AddPowertoolsLogger(options =>
/// {
/// options.LogBuffering = new LogBufferingOptions
/// {
/// Enabled = true,
/// BufferAtLogLevel = LogLevel.Debug
/// };
/// options.LogBuffering.Enabled = true;
/// options.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
/// });
/// </code>
/// </example>
Expand All @@ -188,7 +185,7 @@ public static ILoggingBuilder AddPowertoolsLogger(
UpdateConfiguration(options);

// If buffering is enabled, register buffer providers
if (options.LogBuffering != null)
if (options.LogBuffering?.Enabled == true)
{
// Add a filter for the buffer provider
builder.AddFilter<BufferingLoggerProvider>(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,9 @@ public void BufferSeparation_ConcurrentInvocations_ShouldMaintainSeparateBuffers
Logger.Configure(config =>
{
config.MinimumLogLevel = LogLevel.Debug;
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = false
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = false;
});

for (int e = 0; e < entriesPerInvocation; e++)
Expand Down Expand Up @@ -128,11 +126,9 @@ public void BufferLifecycleIsolation_OverlappingInvocations_ShouldPreserveActive
Logger.Configure(config =>
{
config.MinimumLogLevel = LogLevel.Debug;
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = false
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = false;
});

for (int e = 0; e < entriesPerInvocation; e++)
Expand Down Expand Up @@ -161,11 +157,9 @@ public void BufferLifecycleIsolation_OverlappingInvocations_ShouldPreserveActive
Logger.Configure(config =>
{
config.MinimumLogLevel = LogLevel.Debug;
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = false
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = false;
});

for (int e = 0; e < entriesPerInvocation; e++)
Expand Down Expand Up @@ -224,12 +218,10 @@ public void BufferEvictionIsolation_SizeLimitEviction_ShouldOnlyAffectOwnBuffer(
Logger.Configure(config =>
{
config.MinimumLogLevel = LogLevel.Debug;
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = false,
MaxBytes = smallBufferSize
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = false;
config.LogBuffering.MaxBytes = smallBufferSize;
});

for (int e = 0; e < entriesPerInvocation * 3; e++)
Expand All @@ -256,12 +248,10 @@ public void BufferEvictionIsolation_SizeLimitEviction_ShouldOnlyAffectOwnBuffer(
Logger.Configure(config =>
{
config.MinimumLogLevel = LogLevel.Debug;
config.LogBuffering = new LogBufferingOptions
{
BufferAtLogLevel = LogLevel.Debug,
FlushOnErrorLog = false,
MaxBytes = largeBufferSize
};
config.LogBuffering.Enabled = true;
config.LogBuffering.BufferAtLogLevel = LogLevel.Debug;
config.LogBuffering.FlushOnErrorLog = false;
config.LogBuffering.MaxBytes = largeBufferSize;
});

for (int e = 0; e < entriesPerInvocation; e++)
Expand Down
Loading
Loading