api-ai-replicate

What it does

Describes how to use the Replicate SDK for TypeScript/Node.js to interact with open-source ML models. It covers client setup, executing models synchronously with run(), streaming outputs with stream(), and asynchronous predictions with predictions.create() plus wait() or webhooks. It explains model references as owner/model (latest) or owner/model:version (pinned), and notes file outputs as FileOutput objects with inputs that can be files. It also mentions deployments for cold-start mitigation and training workflows.

How it works

The skill instructs concrete steps including:

• Initialize the Replicate client via code samples using or not using explicit auth, relying on the environment variable REPLICATE_API_TOKEN. It shows how to import Replicate and export the client instance.
• Run predictions synchronously with replicate.run(model, { input }) and handle outputs, where image models yield a FileOutput with .url() and .blob(), and text models yield a string.
• Stream outputs using replicate.stream(model, { input }) and iterate events, processing event.data when event.type is output.
• Use versioning by choosing model references as owner/model or owner/model:version to pin reproducible results.
• Handle file inputs and outputs, including FileOutput objects that must be treated as streams with .url() and .blob(), and support for uploading Buffers or ReadStreams as inputs.
• Use async predictions with replicate.predictions.create({ version, input, webhook, webhook_events_filter }) and then track prediction.id/status; process outputs when webhooks deliver results; validate webhooks with validateWebhook() and a secret.
• Deployments: create predictions on deployments to avoid cold starts and optionally wait for results, using replicate.deployments.predictions.create and replicate.wait.
• Error handling patterns noting automatic retries on 429 and 5xx with a default retry strategy, and status-code specific handling.

When to use it

• Running open-source ML models via Replicate without managing GPU infra
• Streaming real-time LLM output via SSE
• Processing asynchronous predictions with webhook notifications
• Deploying models with custom hardware for latency-sensitive apps
• Training/fine-tuning workflows

What it can touch

• The skill mentions using the Replicate SDK API surface: replicate.run(), replicate.stream(), replicate.predictions.create(), replicate.wait(), replicate.deployments.predictions.create(), and validateWebhook() from the replicate package. It references FileOutput objects for file I/O, including .url() and .blob() methods, and Buffers/ReadStreams for inputs. It relies on environment variables like REPLICATE_API_TOKEN for authentication.

Caveats

• Requires environment-based authentication via REPLICATE_API_TOKEN; token must not be hardcoded.
• Cold starts are possible for infrequently-used models; deployments with min_instances are recommended for latency-sensitive apps.
• Model references should be pinned in production to owner/model:version to ensure reproducible results.
• Webhook handling must validate signatures with validateWebhook(); outputs may arrive as FileOutput objects or other structured data.
• The skill describes serverless, cloud-based execution; local running is not supported.