SCIMT/log_dataset_sizes.out at main · ringger/SCIMT · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
Starting log_dataset_sizes.sh--
Wed Jun 11 10:51:48 AM MDT 2025
-------------------------------
Pipeline/format_SC.sh /home/hatch5o6/Cognate/code/Pipeline/cfg/SC/bho-hi.cfg
Starting train_SC.sh-----------
Wed Jun 11 10:51:48 AM MDT 2025
-------------------------------
Arguments:-
    MODULE_HOME_DIR=/home/hatch5o6/Cognate/code

    SRC=bho
    TGT=hi
    PARALLEL_TRAIN=/home/hatch5o6/Cognate/code/NMT/data/bho-hi/train.csv
    PARALLEL_VAL=/home/hatch5o6/Cognate/code/NMT/data/bho-hi_dev_test/val.csv
    PARALLEL_TEST=/home/hatch5o6/Cognate/code/NMT/data/bho-hi_dev_test/test.csv
    COGNATE_TRAIN=/home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi
    NO_GROUPING=true
    SC_MODEL_TYPE=RNN
    SEED=0
    SC_MODEL_ID=BHO-HI-0
    COGNATE_THRESH=0.5
    COPPERMT_DATA_DIR=/home/hatch5o6/nobackup/archive/CopperMT
    COPPERMT_DIR=/home/hatch5o6/Cognate/code/CopperMT/CopperMT
    PARAMETERS_DIR=/home/hatch5o6/Cognate/code/Pipeline/parameters
    RNN_HYPERPARAMS=/home/hatch5o6/Cognate/code/Pipeline/rnn_hyperparams/default_parameters_rnn_la-it.txt
    BEAM=5
    NBEST=1
    REVERSE_SRC_TGT_COGNATES=false
    VAL_COGNATES_SRC=null
    VAL_COGNATES_TGT=null
    TEST_COGNATES_SRC=null
    TEST_COGNATES_TGT=null
    COGNATE_TRAIN_RATIO=0.80
    COGNATE_TEST_RATIO=0.10
    COGNATE_VAL_RATIO=0.10
-------------------------------


#### DETECT COGNATES ####
removing /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi
creating /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi
creating /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi/cognate
creating /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi/fastalign

############################
# make_SC_training_data.py #
############################
Arguments:-
	-train_csv: /home/hatch5o6/Cognate/code/NMT/data/bho-hi/train.csv
	-val_csv: /home/hatch5o6/Cognate/code/NMT/data/bho-hi_dev_test/val.csv
	-test_csv: /home/hatch5o6/Cognate/code/NMT/data/bho-hi_dev_test/test.csv
	-src_out: /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi/cognate/train.bho
	-tgt_out: /home/hatch5o6/nobackup/archive/data/COGNATE_TRAIN/bho-hi/cognate/train.hi
	-src: bho
	-tgt: hi
------------------------------

random seed: None
MultilingualDataset READING CSV /home/hatch5o6/Cognate/code/NMT/data/bho-hi/train.csv
Reading all lines from /home/hatch5o6/nobackup/archive/data/NLLB/bho_hi/cleaned/src.txt
Reading all lines from /home/hatch5o6/nobackup/archive/data/NLLB/bho_hi/cleaned/tgt.txt