Freelance RAG and prompt engineer (m/f/d)

What it’s about

A language model is only as good as the knowledge it receives at the right moment. Whether an AI assistant gets a delivery time right or makes something up is decided in the chunking, the retrieval and the prompt. That is where we are looking for support: experienced freelancers who have already run retrieval-augmented generation in production. The collaboration is 100% remote, from Germany, Austria or Switzerland, starting from 11 November 2026.

vona is a web agency from Leipzig, remote-first, with four disciplines in one team: AI integration, web and app development, design, and SEO and GEO. We give honest advice, and we agree up front on how a project will be measured. Our clients range from skilled trades and mid-sized companies to startups and established brands across the German-speaking countries.

Here is the kind of project you can expect: for an online retailer with more than 8,000 products, we built an AI customer advisor based on RAG. It accesses the product catalog, FAQ database and order history via API, escalates difficult cases to the team by email and now handles 68% of inquiries on its own. First response time dropped from 18 hours to under 90 seconds. You’ll find more use cases under AI integration.

Your tasks

• Design RAG pipelines for client projects: connect data sources, prepare documents and split them into meaningful chunks, choose embeddings and retrieval methods.
• Implement the pipelines with LangChain, LangGraph or LlamaIndex, depending on what suits the project.
• Write and version system and user prompts, making sure answers stay traceable to their sources and the model openly admits gaps in its knowledge.
• Build test sets of typical customer questions and measure retrieval quality and answer quality before anything goes live and after every change.
• Weigh up with us whether a cloud model such as Claude, GPT or Mistral is the right fit, or whether a local model via Ollama makes more sense.

What you bring

• Several years of software development experience, ideally in Python, and at least one RAG system you have taken into production.
• Confident use of LangChain or LlamaIndex and of at least one vector database or vector search solution.
• A methodical sense of how to measure answer quality rather than relying on gut feeling.
• The ability to explain technical trade-offs so that clients without a technical background can follow them.
• Confident communication in German or English.
• Your own freelance business or self-employed status, based in Germany, Austria or Switzerland.

Nice to have, but not required: experience with LangGraph or agent frameworks such as CrewAI, with Hugging Face, or with running local models.

What we offer

• Real client projects with a clearly defined goal, not feasibility studies that end up in a drawer.
• A dedicated contact person on our side and regular check-ins at an agreed rhythm.
• Access to the AI tools the project needs, such as Claude, Claude Code, ChatGPT or OpenRouter.
• You set your own hours depending on the workload and work entirely from your own setup.
• If the collaboration goes well, the possibility of follow-up projects – a prospect, not a guarantee.

How to apply

Send us a few sentences via the form below about yourself and a RAG project you’re proud of, including what went wrong along the way, if you like. A link to GitHub, your portfolio or LinkedIn is enough; you can skip the formalities. You’ll usually hear back from us within a week.