Skip to content

Commit ed11261

Browse files
authored
feat(actor): adds actor types generation (#1000)
<details> <summary>"backlog"</summary> - [ ] [can optionally be done] language: the language to generate the types for (currently only TypeScript, but ideally structure the code in a way where we can easily add in new languages). char: l - [ ] [optional, nice to have, can be for future work] [input/output/dataset/key-value-store]-schema: path to a specific schema to generate types for - [ ] [don't implement this until we discuss it] add-to-ignore: whether the cli should add the generated files to files like .prettierignore or the biome config, default: false. </details> ### Draft summary `apify actor generate-schema-types --help` ``` Generate TypeScript types from Actor schemas. USAGE $ actor generate-schema-types [path] [--all-optional] [-o <value>] [--strict] ARGUMENTS path Optional path to the input schema file. If not provided, searches default locations. FLAGS --all-optional Mark all properties as optional in generated types. -o, --output=<value> Directory where the generated files should be outputted. Defaults to src/.generated/actor/ to stay within the typical tsconfig rootDir. --strict Whether generated interfaces should be strict (no index signature [key: string]: unknown). DESCRIPTION Generate TypeScript types from Actor schemas. Generates types from the input schema and, when no custom path is provided, also from the dataset schema defined in '.actor/actor.json' under "storages.dataset". Reads the input schema from one of these locations (in priority order): 1. Object in '.actor/actor.json' under "input" key 2. JSON file path in '.actor/actor.json' "input" key 3. .actor/INPUT_SCHEMA.json 4. INPUT_SCHEMA.json Optionally specify custom schema path to use. ``` TO DO: - [x] [slack-feedback] mention in docs that default output path is `src` due to tsconfig - [x] [slack-feedback] `actor generate-schema` -> `generate-schema-types` to keep consistency with `actor validate-schema`? - [x] [slack-feedback] do not export the individual fields, not useful as you can do Input['field'] - this must be done manually, `json-schema-to-typescript` does not support this - [x] [slack-feedback] inline primitive types to make it less verbose (the comments above inlined fields still display in editor) - this must be done manually, `json-schema-to-typescript` does not support this - [x] output types - [x] key-value-store types - [x] how to resolve unknown[] problem? (_As long as the items: { type: string } propagates to string[] type, let's just go that way_.) feedback from @B4nan Must fix - [x] Verify biome-ignore-all actually works or remove it - [x] Handle Promise.all failure gracefully — partial success should still produce output Should fix - [x] Extract schema transform functions to src/lib/schema-transforms.ts - [x] Add unit tests for stripTitles - [x] Handle missing JSON Schema keywords in stripTitles Nice to have - [x] ~~Reduce deep clone overhead~~ lets address this with #1000 (comment) - [x] Deduplicate readOutputSchema logic - [x] Clean up import style, unused defaults
1 parent 821e869 commit ed11261

24 files changed

Lines changed: 1991 additions & 68 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ _mytests
1212
.idea
1313
.zed
1414
docs/changelog.md
15+
.generated
1516

1617
# Yarn files
1718
.yarn/install-state.gz

docs/reference.md

Lines changed: 56 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -419,22 +419,26 @@ DESCRIPTION
419419
Manages runtime data operations inside of a running Actor.
420420
421421
SUBCOMMANDS
422-
actor set-value Sets or removes record into the
423-
default key-value store associated with the Actor run.
424-
actor push-data Saves data to Actor's run
425-
default dataset.
426-
actor get-value Gets a value from the default
427-
key-value store associated with the Actor run.
428-
actor get-public-url Get an HTTP URL that allows
429-
public access to a key-value store item.
430-
actor get-input Gets the Actor input value from
431-
the default key-value store associated with the Actor
432-
run.
433-
actor charge Charge for a specific event in
434-
the pay-per-event Actor run.
435-
actor calculate-memory Calculates the Actor’s dynamic
436-
memory usage based on a memory expression from
437-
actor.json, input data, and run options.
422+
actor set-value Sets or removes record
423+
into the default key-value store associated with
424+
the Actor run.
425+
actor push-data Saves data to Actor's
426+
run default dataset.
427+
actor get-value Gets a value from the
428+
default key-value store associated with the Actor
429+
run.
430+
actor get-public-url Get an HTTP URL that
431+
allows public access to a key-value store item.
432+
actor get-input Gets the Actor input
433+
value from the default key-value store associated
434+
with the Actor run.
435+
actor charge Charge for a specific
436+
event in the pay-per-event Actor run.
437+
actor calculate-memory Calculates the Actor’s
438+
dynamic memory usage based on a memory expression
439+
from actor.json, input data, and run options.
440+
actor generate-schema-types Generate TypeScript
441+
types from Actor schemas.
438442
```
439443
440444
##### `apify actor calculate-memory`
@@ -491,6 +495,42 @@ FLAGS
491495
charging without actually charging
492496
```
493497
498+
##### `apify actor generate-schema-types`
499+
500+
```sh
501+
DESCRIPTION
502+
Generate TypeScript types from Actor schemas.
503+
504+
Generates types from the input schema and, when no custom path is provided,
505+
also from the dataset schema defined in '.actor/actor.json' under
506+
"storages.dataset".
507+
508+
Reads the input schema from one of these locations (in priority order):
509+
1. Object in '.actor/actor.json' under "input" key
510+
2. JSON file path in '.actor/actor.json' "input" key
511+
3. .actor/INPUT_SCHEMA.json
512+
4. INPUT_SCHEMA.json
513+
514+
Optionally specify custom schema path to use.
515+
516+
USAGE
517+
$ apify actor generate-schema-types [path]
518+
[--all-optional] [-o <value>] [--strict]
519+
520+
ARGUMENTS
521+
path Optional path to the input schema file. If not provided, searches
522+
default locations.
523+
524+
FLAGS
525+
--all-optional Mark all properties as optional in
526+
generated types.
527+
-o, --output=<value> Directory where the generated files
528+
should be outputted. Defaults to src/.generated/actor/ to
529+
stay within the typical tsconfig rootDir.
530+
--strict Whether generated interfaces should be
531+
strict (no index signature [key: string]: unknown).
532+
```
533+
494534
##### `apify actor get-input`
495535
496536
```sh

package.json

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@
100100
"istextorbinary": "~9.5.0",
101101
"jju": "~1.4.0",
102102
"js-levenshtein": "^1.1.6",
103-
"lodash.clonedeep": "^4.5.0",
103+
"json-schema-to-typescript": "^15.0.4",
104104
"mime": "~4.1.0",
105105
"open": "~11.0.0",
106106
"rimraf": "~6.1.0",
@@ -127,7 +127,6 @@
127127
"@types/express": "^5.0.0",
128128
"@types/jju": "^1.4.5",
129129
"@types/js-levenshtein": "^1",
130-
"@types/lodash.clonedeep": "^4",
131130
"@types/mime": "^4.0.0",
132131
"@types/node": "^24.0.0",
133132
"@types/semver": "^7.5.8",

scripts/generate-cli-docs.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ const categories: Record<string, CommandsInCategory[]> = {
3434
{ command: Commands.actor },
3535
{ command: Commands.actorCalculateMemory },
3636
{ command: Commands.actorCharge },
37+
{ command: Commands.actorGenerateSchemaTypes },
3738
{ command: Commands.actorGetInput },
3839
{ command: Commands.actorGetPublicUrl },
3940
{ command: Commands.actorGetValue },

src/commands/_register.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import type { BuiltApifyCommand } from '../lib/command-framework/apify-command.j
22
import { ActorIndexCommand } from './actor/_index.js';
33
import { ActorCalculateMemoryCommand } from './actor/calculate-memory.js';
44
import { ActorChargeCommand } from './actor/charge.js';
5+
import { ActorGenerateSchemaTypesCommand } from './actor/generate-schema-types.js';
56
import { ActorGetInputCommand } from './actor/get-input.js';
67
import { ActorGetPublicUrlCommand } from './actor/get-public-url.js';
78
import { ActorGetValueCommand } from './actor/get-value.js';
@@ -77,6 +78,7 @@ export const actorCommands = [
7778
ActorGetInputCommand,
7879
ActorChargeCommand,
7980
ActorCalculateMemoryCommand,
81+
ActorGenerateSchemaTypesCommand,
8082

8183
// top-level
8284
HelpCommand,

src/commands/actor/_index.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { ApifyCommand } from '../../lib/command-framework/apify-command.js';
22
import { ActorCalculateMemoryCommand } from './calculate-memory.js';
33
import { ActorChargeCommand } from './charge.js';
4+
import { ActorGenerateSchemaTypesCommand } from './generate-schema-types.js';
45
import { ActorGetInputCommand } from './get-input.js';
56
import { ActorGetPublicUrlCommand } from './get-public-url.js';
67
import { ActorGetValueCommand } from './get-value.js';
@@ -21,6 +22,7 @@ export class ActorIndexCommand extends ApifyCommand<typeof ActorIndexCommand> {
2122
ActorGetInputCommand,
2223
ActorChargeCommand,
2324
ActorCalculateMemoryCommand,
25+
ActorGenerateSchemaTypesCommand,
2426
];
2527

2628
async run() {

0 commit comments

Comments
 (0)