Skip to content

Commit cb3dac7

Browse files
fix: use args.context_length for packing, lower save_steps
1 parent 9e5db58 commit cb3dac7

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

training/train_extended_context.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -291,8 +291,8 @@ def tokenize(example):
291291
gradient_checkpointing=True,
292292
gradient_checkpointing_kwargs={"use_reentrant": False},
293293
logging_steps=10,
294-
save_steps=100,
295-
eval_steps=100,
294+
save_steps=50,
295+
eval_steps=50,
296296
save_total_limit=2,
297297
optim="adamw_torch",
298298
report_to="none",

0 commit comments

Comments
 (0)