When we shipped AI Result Evaluation a couple of weeks ago, the data model was deliberately source-agnostic — every batch is identified by a sourceType and a sourceId pointer instead of a hard foreign key into one specific table. But only two source types actually existed: verification_pipeline_run and lt_pipeline_run. Both required sourceId to resolve to a row Crowdee itself had created. If your fraud-detection model flagged a transaction and you wanted a crowd panel to check whether its explanation actually held up, there was nowhere to plug that in. We said as much at the time — plainly, in the docs and in that launch post — because the honest answer was "not yet."
It's not "not yet" anymore.
sourceType: "external_submission" is now a third option, and it works differently from the other two on purpose: there's no pre-existing run to look up, because the AI that produced the result was never inside Crowdee in the first place. So instead of resolving an ID, there's a dedicated endpoint that takes the actual output directly:
curl -X POST https://api.crowdee.ai/v2/projects/{projectId}/ai-output-evaluations/external \
-H "X-API-Key: crw_YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"modelName": "Acme Fraud Detector v3",
"verdict": "likely fraudulent",
"explanation": "Transaction pattern matches known card-testing behaviour: 14 micro-charges across 6 minutes to unrelated merchants.",
"confidence": 87,
"evidenceUrls": ["https://internal.acme.example/case/8841"]
}'
Model name, verdict, explanation, an optional confidence score, optional evidence links. That's it — no template to design, no project files to upload first. The crowd task gets generated the same way it always does, showing your submitted verdict and explanation, and asking the same four questions: is this clear, is it backed by evidence, is it actionable, and how much bias risk does it carry.
The obvious way to store an external submission would be a new table — something like externalAiSubmissions with a column per field. We didn't do that. Crowdee already has a generic model for exactly this shape of data: inputDataSets and inputDataLists, originally built so crowd jobs can attach structured, per-variant input to worker tasks. A single external submission is just an input data set with one variant: one list named verdict, one named explanation, one named confidence if you sent one, and so on. sourceId on the resulting evaluation batch is the id of that data set — the exact same id you'd get back if you'd created it through the regular input-data API yourself.
This wasn't a difficult call to make. The alternative — a dedicated table that exists purely to hold verdict/explanation/confidence strings — would have duplicated infrastructure we already had, for no real benefit. When an existing generic model already fits, adding a narrower one next to it is just a second thing to keep in sync.
This is the part we were most careful about: none of the endpoints that already worked needed to change. GET /v2/ai-output-evaluations, the results endpoint, the feedback log, the credit-blocking logic, the transparency-rating widget the crowd actually sees — all of it was written against sourceType/sourceId from day one, precisely so that a third source type would be additive. The only new code is the part that's genuinely new: an endpoint to accept output that doesn't come from us, and the input-data rows it writes on your behalf.
Same formula as every other evaluation: 15 credits per crowd response, 3 responses by default, so 45 credits per submission unless you ask for more responses. There's no separate external-submission fee — from a cost and infrastructure standpoint, evaluating your own AI's output is exactly as expensive as evaluating one of ours. See the AI Output Evaluation docs for the full field reference, or ask the MCP server's create_external_ai_output_evaluation tool to do it for you.
Share this article: