Dear Runway API Team,
First - thank you for adding the best models to the Runway API.
Question - Why are the newer models not consistent with previous model API implementations? This requires model specific workarounds in the code, compared to previous model integrations.
The following schema workarounds needed to be taken into account with newer models mentioned - and unclear from API docs:
For Image to Video:
-
ratio : always used before in pixels sizes across all models, now some use the actual ratios, but grok_imagine_1_5 and happyhorse_1_0 fail if added for image2video gen, but required in text2video gen?
-
resolution : This has been added to the newer models, but happyhorse_1_0 and hailuo3 require the resolution uppercase and while grok_imagine_1_5 requires it in lowercase - else it fails.
First/ Last frame support:
- Newer models that support multi references images has moved to text2video as references, but in theory possible as image to video option with position added, but not supported.
Reference Images, Video and Audio:
- Some AI providers have moved to new category - Ref2Image or Ref2Video and cleaner code. Adding references to text2video not sure if that makes sense.
Final Thought for today:
What might be the way forward would be to only have one Image- and Video gen endpoint. With the API using the input schema with text plus references in a unified way. This will allow models to be defined based on elements supported and used in context as needed.
Dear Runway API Team,
First - thank you for adding the best models to the Runway API.
Question - Why are the newer models not consistent with previous model API implementations? This requires model specific workarounds in the code, compared to previous model integrations.
The following schema workarounds needed to be taken into account with newer models mentioned - and unclear from API docs:
For Image to Video:
ratio : always used before in pixels sizes across all models, now some use the actual ratios, but grok_imagine_1_5 and happyhorse_1_0 fail if added for image2video gen, but required in text2video gen?
resolution : This has been added to the newer models, but happyhorse_1_0 and hailuo3 require the resolution uppercase and while grok_imagine_1_5 requires it in lowercase - else it fails.
First/ Last frame support:
Reference Images, Video and Audio:
Final Thought for today:
What might be the way forward would be to only have one Image- and Video gen endpoint. With the API using the input schema with text plus references in a unified way. This will allow models to be defined based on elements supported and used in context as needed.