turning off the tqdm progress bar when sampling

[zhongzhenzhen]

Problem Description

I'm working on a reinforcement learning project for a game agent, and during training I use SDV to generate synthetic data for each step. Because the training loop runs many iterations, SDV prints a tqdm progress bar every time it samples, which overwhelms my logs and makes it hard to see the actual training output.

Expected behavior

Adding a verbose parameter to the sample to give the option of turning off the progress bar.

[npatki]

Thanks for filing this feature request @zhongzhenzhen. Our team will discuss and add to the queue.

Workaround

Over Slack, we discussed a possible workaround of disabling the tqdm library globally to get rid of all progress bars. (Following either this StackOverflow response or this GitHub issue).

You mentioned that you ran into problems when doing this. Could you elaborate? Did these workarounds not produce the desired result? Or are you hoping to keep tqdm around for other parts of your workflow (and just disable it for SDV)?

[zhongzhenzhen]

Never mind, I found another solution that can successfully turning off sdv progress bar for conditional sampling:

from contextlib import contextmanager

class SilentTqdm:
    def __init__(self, *args, **kwargs):
        self.n = 0
        self.total = kwargs.get('total', 100)
        self.disable = True
    def __enter__(self):
        return self
    def __exit__(self, *args):
        return False
    def update(self, n=1):
        self.n += n
    def close(self):
        pass
    def __iter__(self):
        return iter([])
    def set_description(self, *args, **kwargs):
        pass
    def set_postfix(self, *args, **kwargs):
        pass
    def refresh(self, *args, **kwargs):
        pass

# Patch tqdm module before SDV imports it
try:
    import tqdm
    tqdm.tqdm = SilentTqdm
    # Patch in sys.modules
    if 'tqdm' in sys.modules:
        sys.modules['tqdm'].tqdm = SilentTqdm
    if 'tqdm.auto' in sys.modules:
        sys.modules['tqdm.auto'].tqdm = SilentTqdm
except:
    pass

# Now import SDV
from sdv.sampling import DataFrameCondition

# Patch tqdm in SDV modules immediately after import
try:
    import sdv.sampling.tabular as sdv_tabular
    if hasattr(sdv_tabular, 'tqdm'):
        sdv_tabular.tqdm = SilentTqdm
except:
    pass

try:
    import rdt.transformers.base as rdt_base
    if hasattr(rdt_base, 'tqdm'):
        rdt_base.tqdm = SilentTqdm
except:
    pass

@contextmanager
def suppress_output():
    """Context manager to suppress both stdout and stderr"""
    with open(os.devnull, 'w') as devnull:
        old_stdout = sys.stdout
        old_stderr = sys.stderr
        sys.stdout = devnaull
        sys.stderr = devnull
        try:
            yield
        finally:
            sys.stdout = old_stdout
            sys.stderr = old_stderr

[npatki]

Awesome! Thanks for sharing @zhongzhenzhen -- I'm sure this will help out anyone else who's running into the same problem.