Broad reactions / questions / pain-points I had while learning how to use cog

[dashstander]

This is formatted as “question I had” italics, then answers / comments underneath it.

• Do I need to add cog as a dependency?
  • No, it’s added automatically in the build process
  • But it’s a little annoying to develop in an IDE, have the IDE not know about the cog image, and see that little squiggly yellow line saying that it doesn’t know where to import cog from
• How does the signature to a cog Predictor.predict method actually work? What gets returned and how?
  • It looks like one of the main things to pass in/out are Path objects which point to actual files. Which is to say that you need to go in and load the file
  • Honestly this feels a little awkward that I have to write to disk for this to work.
  • Why can’t I just return an image object (if there’s a type annotation?) or if it needs to be a file, why not a file-like object?
• What gets displayed while the prediction is running? Is there a way to have a progress bar?
  • I still don’t know the answer to this.
• When it says that docker is a requirement, do I also need nvidia-container-toolkit if I want to build an image for GPU?
  • I certainly needed to to get cog to work on my computer
• Can I run a jupyter notebook from within the cog image?
  • Apparently “yes” according to @andreas Jansson, but it’s a long and complicated command
  • It would be really nice for development if this were easier and/or if that long command were documented
• Can I access a shell from within the container?
  • Also “yes”, just do cog run bash
  • I would definitely like this called out in the documentation (TODO: make documentation PR)
• Do I need to rebuild the docker image every time I change a file?
  • Answer appears to be “yes”?
  • It’d be nice to have have “hot reloading”, where I can be developing, then do cog run pytest tests , see what fails, make some edits, run it again, etc...
• Where is the Dockerfile that cog creates?
  • I have no idea
  • I think if I knew where it was I could point my IDE to it (the dockerfile) and use the tooling VSCode has to work “within” the container
• Can I integrate my container with docker compose?
  • Also have no idea
  • This would be really cool. It would be nice to be able to use docker compose to smoothly integrate a model with a database or something like redis.
  • Use cases of this would be for retrieval models and having the vector index and/or the other images in a database

[anish-sahoo]

This is a broad feedback thread; many points have since been addressed in docs and CLI, and the remaining items (watch/hot-reload mode, Docker Compose integration, public Dockerfile command, prediction progress bars) will be addressed in future focused issues.