Know where your examples go.
When you use the Playground
Playground examples are supplied by JevBench. Their Jev scores were recorded in advance. Opening a scenario does not send a new request to Jev or OpenRouter. If you choose to run the Playground live, the state and decisions shown in the editor are sent through JevBench to OpenRouter and its Jev provider.
When you explore the demo
The example contains fictional business inputs and illustrative scores. Exploring thresholds does not send rows to Jev. Metrics are calculated in your browser.
When you prepare your own decision
A signed-in user can save one Jev decision: the question and the labeled examples. Saving does not send those examples to Jev. CSV files are parsed in the browser, and the original file is not uploaded.
When you run your own data
Jev evaluation happens only when you start an early look, a baseline, a question revision, or fresh validation. Saving the decision does not send examples to Jev. One decision can score up to 10 examples before payment. Revisions, fresh checks, and any examples past those 10 need a verified purchase. Deleting the decision does not reset that early look. Your question, labels, dataset, scores, and run history are stored in your account’s database. Each input and the question are sent from the JevBench server to OpenRouter and its Jev provider.
Expected human labels are used for analysis and are never sent to Jev. OpenRouter and the provider’s data policies apply to live requests. The provider API key remains on the server. The same question and examples are stored if you send them through the API or MCP, and they are still sent to Jev only when a run advances. A personal API key is shown once and stored only as a hash.
Deleting a saved decision
You can delete a decision from your account. That removes its question, examples, runs, and scores from the active database. Provider retention and database backups follow their own policies.
Product analytics
We use PostHog to measure which pages and recorded Playground scenarios people explore, whether they inspect mistakes, and whether they start preparing their own decision. Events contain counts, cutoff values, and scenario identifiers. They do not contain your question, examples, CSV, result text, API keys, or Checkout addresses. Public visitors are measured with anonymous browser identifiers. After you sign in, later events use an internal account id. We do not send your email address or GitHub identity.
Session replay
PostHog records visits to the public home page and Playground to help us understand navigation and layout. The Playground workspace, including state, decisions, results, and request previews, is blocked from replay. Input values are masked. Recording stops on account, checkout, saved-decision, and other pages.
Provider policy
JevBench is an independent evaluation tool and is not affiliated with or endorsed by TypeSafe AI.