Your RAG is only as fresh as your last re-index.
Remac keeps your vector database in sync with every row change - in real time. A product description changes in Postgres, the embedding in Pinecone updates seconds later. No cron. No drift. No stale context feeding hallucinations.
Stale context is a data pipeline problem.
From row change to fresh embedding — in one config.
The embedding transformer is just another step in the stream. Declare the provider, the input template, and the vector sink - Remac handles capture, embedding, caching, and upsert.
streams:- id: catalog-ragsource:type: postgressettings:replication_slot: remac_ragmiddlewares:filters:- type: tablesettings:include: [products]transformers:- type: embeddingsettings:provider: openaimodel: text-embedding-3-smallinput: "{{ name }} - {{ description }}"cache: true # skip unchangedsinks:- type: pineconeenabled: truesettings:index: catalognamespace: products
CDC decodes the row change off the transaction log (WAL in this case with a Postgres source) — insert, update, or delete.
The transformer renders the input template and calls your provider — cache skips unchanged content.
The vector lands in your store within seconds — context stays current, no cron.
Every vector database is just a sink.
Same architecture as Kafka or S3 — point the stream at your store and the embedding transformer handles the rest.
Fresh data, all the way down the AI stack.
Every transaction that touches a feature-relevant table updates the store immediately. Model serving always reads the freshest features — no waiting for the next batch.
Continuous archival of production changes to S3 as JSONL, directly queryable by any engine that reads it. Your training pipeline is a YAML file, not an Airflow DAG.
Cron re-index vs Remac sync.
A bolt-on, not a second product.
Same binary, same pipeline — add it to any paid tier and the AI capabilities switch on. All your regular CDC keeps working, gated by the license, not a separate deploy.
Calls OpenAI today; attaches the vector to the event. Cohere, Google Gemini, Mistral, Voyage AI, and more on the roadmap.
Qdrant supported today. Pinecone, Weaviate, Milvus, pgvector, ChromaDB, and more on the roadmap — each just a sink type.
Embedding latency, write throughput, and sync lag at a glance.
Skips re-embedding unchanged content — saves real API spend.
From RAG to training pipelines.
Here’s where teams start. The pipeline doesn’t care what’s downstream.
Keep the vector store in lockstep with the source so the assistant never cites stale data.
Stream feature-relevant changes to the store so models serve on the freshest features.
Continuous production-data archival to S3 as JSONL — training sets that build themselves.
Fan one source out to many stores and services — without owning the pipeline.
AI Pack questions
Which embedding models can I use?+
Does it cache embeddings?+
What's the sync latency?+
Can I keep everything self-hosted?+
Which vector databases are supported?+
Stop re-indexing. Start remacing.
Cron jobs and full re-index runs are a workaround. Real-time CDC is the fix.