Serial flag python by sven-oly · Pull Request #430 · unicode-org/conformance

sven-oly · 2025-03-31T23:45:49Z

sffc · 2025-04-01T00:43:41Z

There is an error in CI:

~/work/conformance/conformance
~/work/conformance/conformance/schema ~/work/conformance/conformance
Traceback (most recent call last):
  File "/home/runner/work/conformance/conformance/schema/check_schemas.py", line 153, in <module>
    main(sys.argv)
  File "/home/runner/work/conformance/conformance/schema/check_schemas.py", line 99, in main
    validator.options = SchemaArgs(args).getOptions()
                        ^^^^^^^^^^
NameError: name 'SchemaArgs' is not defined. Did you mean: 'schemaArgs'?

schema/check_generated_data.py

…flag_python Update comment

sven-oly · 2025-04-01T01:43:03Z

I'll work on this now. I pushed too soon!

sven-oly · 2025-09-04T23:28:35Z

#386

sven-oly

revert the files:

PluralRulesTest.java
datetime_fmt.py

with @echeran

sven-oly · 2025-09-05T22:41:10Z

schema/check_schemas.py

-    num_processors = multiprocessing.cpu_count()
-    logging.info('Schema validation: %s processors for %s schema validations', num_processors, len(file_names))
+def validate_all_schema(validator, file_names):
+    if not validator.options.run_serial:


flip the "then" and "else" branches so that you can avoid using not in your condition test.

sven-oly · 2025-09-05T22:42:25Z

schema/check_test_output.py

+
+    # file_base + output_path
+    test_output_path = schema_options.schema_base
+    print('!!! TEST OUTPUT PATH %s' % (test_output_path))


no prints. only logging, and if necessary

sven-oly · 2025-09-05T22:44:18Z

schema/schema_validator.py


        # Check for all the possible files
        json_file_pattern = os.path.join(self.test_data_base, '*', '*.json')
+        logging.info('JSON FILE_PATTERN: %s', json_file_pattern);


for here and elsewhere (above & below): change log level to debug if this is not useful to see in a log file (ex: looking at a CI run)

sven-oly · 2025-09-05T22:45:02Z

schema/schema_validator.py

-            result = p.map(self.check_test_data_against_schema, schema_test_data)
-        return result
+    def check_all_test_data_schema(self, schema_test_data):
+        if not self.run_serial:


sven-oly · 2025-09-05T22:46:57Z

schema/schema_validator.py

+            for test_data in schema_test_data:
+                results.append(self.check_test_data_against_schema(test_data))


could turn the instantiation + initialization of results into a FP-style map. Not sure how Python does that.

sven-oly · 2025-09-05T22:48:17Z

schema/schema_validator.py

+            return results, test_validation_plans
+        else:
+            logging.info('JSON test output serially validation on %s files!', len(test_validation_plans))
+            for test_data in test_validation_plans:


ditto. also, does results need to be initialized before you call .append on it?

sven-oly · 2025-09-05T22:48:22Z

schema/schema_validator.py

-        num_processors = mp.cpu_count()
-        logging.info('JSON test output validation: %s processors for %s plans', num_processors,
-                     len(test_validation_plans))
+        if not self.run_serial:


sven-oly · 2025-09-05T22:50:18Z

schema/schema_validator.py

-        results = self.parallel_check_test_data_schema(schema_test_info)
+        results = self.check_all_test_data_schema(schema_test_info)

        for result_data in results:


can replace the instantiation, for loop, and if with an FP-style map + filter.

CLAassistant · 2025-11-05T02:31:57Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ sven-oly
❌ Craig Cornelius

Craig Cornelius seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

sven-oly added 2 commits March 4, 2025 14:34

Starting to add serial flag for schema testing

e755a6c

Merge branch 'unicode-org:main' into serial_flag_python

febe481

sven-oly requested review from echeran and sffc March 31, 2025 23:45

sffc reviewed Apr 1, 2025

View reviewed changes

schema/check_generated_data.py Outdated Show resolved Hide resolved

sven-oly added 3 commits March 31, 2025 18:11

--a

c6f5b22

Merge remote-tracking branch 'upstream/main' into serial_flag_python

7da5f8f

Merge remote-tracking branch 'origin/serial_flag_python' into serial_…

cd0a3e1

…flag_python Update comment

sven-oly added 5 commits August 29, 2025 10:41

Restructuring command line arguments for Schema processing

4a3f16e

Merge remote-tracking branch 'upstream/main' into serial_flag_python

f322376

Adding serial running to schema validation

4571c4f

Update generator with schema_base

4fe9432

Simplifying some code

b79ab7a

sven-oly mentioned this pull request Sep 4, 2025

Add global flags to set serial execution of python scripts #386

Open

sven-oly added 2 commits September 4, 2025 17:28

Fix path for check_schemas.py

de30571

Synch

71bd8d6

sven-oly commented Sep 5, 2025

View reviewed changes

sven-oly added 5 commits September 5, 2025 23:17

Remove diff

fd64f76

Remove file

054a861

Revising as suggested

de19fb8

Fix error in return values

a630a03

Reply to gemini comment

c3f75c1

echeran linked an issue Sep 8, 2025 that may be closed by this pull request

Add global flags to set serial execution of python scripts #386

Open

sven-oly assigned echeran and sffc Sep 13, 2025

sffc self-requested a review September 29, 2025 22:19

sffc approved these changes Oct 9, 2025

View reviewed changes

Merge branch 'main' into serial_flag_python

7491a1f

sffc approved these changes Oct 10, 2025

View reviewed changes

Remove extra output

a08251f

		for test_data in schema_test_data:
		results.append(self.check_test_data_against_schema(test_data))

Uh oh!

Conversation

sven-oly commented Mar 31, 2025

Uh oh!

sffc commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

sven-oly commented Apr 1, 2025

Uh oh!

sven-oly commented Sep 4, 2025

Uh oh!

sven-oly left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sffc commented Apr 1, 2025 •

edited

Loading