Welcome to the APM Challenge

Test your APM / monitoring tool against real-world security scenarios.
Can your agent detect what's wrong?

sina_player — APM Scenario Player

100% success ratio on all 23 challenges. sina_player is an agentless APM scenario player built on Chrome DevTools Protocol (CDP). No browser plugin, no OS agent, no injected JavaScript — just pure CDP automation through sina_web browser pooling.

172 Selenium-compatible functions across 12 categories (navigation, clicks, inputs, waits, assertions, frames, windows, cookies, storage, downloads, CDP, utilities)
CDP Trusted keyboard & mouse events via sinaTrustedType() — passes isTrusted checks that defeat Selenium send_keys
AUTO Auto-detection: popup dismissal (sinaAcceptPopup), CAPTCHA solving (reCAPTCHA, hCaptcha, Turnstile, image CAPTCHA), Shadow DOM piercing
PY Editable Python sources — scenarios are plain .py files you can version, review, and customize with full Python standard library access
SCHED Stateful scheduling via sina_scheduler — FIFO/LIFO pool management, cyclic execution, retry policies, and multi-node distribution
250x sina_web browser pooling — up to 250x throughput vs. single-browser execution, with session isolation and automatic cleanup

New challenges suggested by users may reveal edge cases — we update sina_player when they do. The 100% score reflects current challenges; community suggestions help us improve.

At the bottom of this page please find our estimate of the success ratio for the known APM (Commercial and Open Source).
Please note that this is an estimate from reading the available documentation and forums. Please prove us wrong and have fun coding/designing/testing the scenarios on each challenge page.

Challenges

1. TLS 1.1 Detection

A website serving content over TLS 1.1 — a deprecated, insecure protocol. Can your APM agent detect this vulnerability?

https://tls11.apm-challenge.sinainsight.eu:4446/
Go
This URL will NOT open in a browser. Modern browsers reject TLS 1.1 connections (ERR_SSL_VERSION_OR_CIPHER_MISMATCH). Point your APM agent at the URL above and check if it reports the TLS vulnerability.

Verify with: curl -svk https://tls11.apm-challenge.sinainsight.eu:4446/
Expected: TLS alert, protocol version — server only accepts TLS 1.0/1.1.

Register or use anonymous mode to submit results.

2. Cookie / RGPD Consent Popup

A page with a RGPD/cookie consent modal that blocks access to the content. Your APM agent must accept the cookies or privacy preferences, then verify that a signature text is present on the page behind the popup.

https://apm-challenge.sinainsight.eu/challenge/rgpd
Go
Your agent must: 1) Detect and dismiss the cookie/RGPD consent popup (click "Accept All"). 2) Verify the signature text "SINAINSIGHT-RGPD-VERIFIED" is visible on the page after acceptance.

Register or use anonymous mode to submit results.

3. Anti-Robot Form (isTrusted Keystroke)

A login form protected by keystroke verification. The fields are disabled until a real keyboard event (isTrusted: true) is detected. Selenium's send_keys() and JavaScript injection fail — your agent must use CDP-level or OS-level keystrokes to unlock the form and reveal the signature text.

https://apm-challenge.sinainsight.eu/challenge/antirobot
Go
Your agent must: 1) Type into the disabled form using real keyboard events (not send_keys). 2) Verify the signature text "SINAINSIGHT-ANTIROBOT-VERIFIED" is visible after the form unlocks.

Register or use anonymous mode to submit results.

4. Shadow DOM Encapsulation

A page using Web Components with Shadow DOM. The signature text is hidden inside an encapsulated shadow root — standard selectors (querySelector, find_element) cannot reach it. Your agent must traverse the shadow boundary to read the content and click a button inside the shadow tree.

https://apm-challenge.sinainsight.eu/challenge/shadowdom
Go
Your agent must: 1) Access the Shadow DOM of the <challenge-box> element. 2) Click "Reveal Signature" inside the shadow root. 3) Verify "SINAINSIGHT-SHADOWDOM-VERIFIED" is present.

Register or use anonymous mode to submit results.

5. Delayed DOM Content

A page where the signature text is injected into the DOM after a 5-second delay via JavaScript. An agent that reads the page immediately will see nothing — it must wait for the dynamic content to appear.

https://apm-challenge.sinainsight.eu/challenge/delayed
Go
Your agent must: 1) Navigate to the page. 2) Wait for the signature to appear in the DOM (~5 seconds). 3) Verify "SINAINSIGHT-DELAYED-VERIFIED" is present.

Register or use anonymous mode to submit results.

6. Scrambled Form

A registration form whose fields are shuffled into a random order on every page load. The field IDs never change — only the visual position does. Your agent must target fields by ID or name, not by positional selectors.

https://apm-challenge.sinainsight.eu/challenge/scrambled
Go
Your agent must: 1) Fill all 5 form fields using their stable IDs (not position). 2) Submit the form. 3) Verify "SINAINSIGHT-SCRAMBLED-VERIFIED" on the success page.

Register or use anonymous mode to submit results.

7. Dynamic IDs (Session Tokens)

A contact form where session-specific tokens are embedded in every element ID. IDs change on every page load (e.g. dynid-input-email-s7f3a2b). Fields are also shuffled. Your agent must use partial ID matching or name attributes.

https://apm-challenge.sinainsight.eu/challenge/dynid
Go
Your agent must: 1) Handle dynamic IDs with session tokens embedded in element attributes. 2) Fill all 5 fields using partial ID matching or name attributes. 3) Submit and verify "SINAINSIGHT-DYNID-VERIFIED".

Register or use anonymous mode to submit results.

8. Annoying Recurring Popup

A page with a blocking modal popup that appears immediately and reappears every 3 seconds after dismissal. The signature text loads after a 5-second delay, hidden behind the popups. Your agent must dismiss popups repeatedly while waiting for the content.

https://apm-challenge.sinainsight.eu/challenge/popup
Go
Your agent must: 1) Dismiss the recurring modal popups (they reappear every 3s). 2) Wait for the signature to appear (~5s delay). 3) Verify "SINAINSIGHT-POPUP-VERIFIED" is present.

Register or use anonymous mode to submit results.

9. PDF Download & Verify

The signature is not on a web page — it is inside a downloadable PDF document. Your agent must download the file, parse the PDF format, and extract the signature text.

https://apm-challenge.sinainsight.eu/challenge/pdf
Go
Your agent must: 1) Download the PDF from the challenge page. 2) Parse the PDF and extract text content. 3) Verify "SINAINSIGHT-PDF-VERIFIED" is inside the PDF.

Register or use anonymous mode to submit results.

10. JSON API & JSONPath

The signature is buried deep inside a JSON API response with nested objects and arrays. Your agent must download the JSON, parse it, and query a specific JSONPath to extract the value. Flat text search won't help — you need structured navigation.

https://apm-challenge.sinainsight.eu/challenge/json
Go
Your agent must: 1) Download the JSON from the challenge page. 2) Query $.system.diagnostics[*].metadata.signature. 3) Verify "SINAINSIGHT-JSON-VERIFIED" is in the results.

Register or use anonymous mode to submit results.

11. Image Integrity Verification

A PNG image is served from the challenge endpoint. Your agent must download the image binary content, compute its SHA-256 hash, and verify it matches the expected value. This simulates file integrity monitoring for tampered assets.

https://apm-challenge.sinainsight.eu/challenge/image
Go
Your agent must: 1) Download the PNG image. 2) Compute SHA-256 hash of the file. 3) Verify it matches the expected hash shown on the challenge page.

Register or use anonymous mode to submit results.

12. Nested Iframe Navigation

The signature is hidden inside a doubly-nested iframe structure. The top page contains an iframe, which contains another iframe with the target text. Standard DOM queries on the top document find nothing — your agent must switch frame context.

https://apm-challenge.sinainsight.eu/challenge/iframe
Go
Your agent must: 1) Switch into #iframe-level1. 2) Switch into #iframe-level2 inside it. 3) Verify "SINAINSIGHT-IFRAME-VERIFIED" in the inner document.

Register or use anonymous mode to submit results.

13. Value-Loaded Form (.value Property)

A form whose fields are populated via JavaScript .value assignment after staggered API-like delays. The HTML attribute remains empty — only the live DOM property holds the data. Your agent must read .value, not getAttribute('value').

https://apm-challenge.sinainsight.eu/challenge/valueform
Go
Your agent must: 1) Wait for all form fields to be populated (~3s staggered loading). 2) Read the Verification Code field's .value property. 3) Verify "SINAINSIGHT-VALUEFORM-VERIFIED" is the value.

Register or use anonymous mode to submit results.

14. CSV Log Analysis

A CSV access log file with 500+ entries. The signature is hidden as a specific entry among normal traffic. Your agent must download the file and use regex to find the anomaly.

https://apm-challenge.sinainsight.eu/challenge/csv
Go
Your agent must: 1) Download the CSV log file. 2) Search using regex pattern SINAINSIGHT-CSV-[A-Z0-9]+. 3) Verify "SINAINSIGHT-CSV-VERIFIED" is found in the log.

Register or use anonymous mode to submit results.

15. Multi-Window / Tab Navigation

The signature is in a popup window opened by a button click. Your agent must detect the new window handle, switch to it, read the content, and switch back. The signature does NOT exist in the original page.

https://apm-challenge.sinainsight.eu/challenge/multiwindow
Go
Your agent must: 1) Click the button to open the signature window. 2) Switch to the new window handle. 3) Verify "SINAINSIGHT-MULTIWINDOW-VERIFIED" in the popup.

Register or use anonymous mode to submit results.

16. YAML Configuration Audit

A YAML configuration file simulating a production cluster setup with deeply nested structures, multi-line strings (TLS certificates), special characters (colons, braces, brackets, quotes), and Unicode text. The signature is buried in an array of monitoring alert objects. Your agent must parse YAML correctly — regex on raw text will break on quoted strings and multi-line blocks.

https://apm-challenge.sinainsight.eu/challenge/yaml
Go
Your agent must: 1) Download the YAML configuration file.
2) Parse the YAML and navigate to monitoring.alerts[*].metadata.signature.
3) Verify "SINAINSIGHT-YAML-VERIFIED" is present.

Register or use anonymous mode to submit results.

17. Record & Replay Workflow

A multi-step business workflow that must be completed in sequence: read a dynamic code and type it, select the correct protocol from a dropdown, check matching services, accept a JavaScript confirm dialog, and click a delayed button. The signature only appears after all 5 steps succeed — this is what a recorder captures.

https://apm-challenge.sinainsight.eu/challenge/recorder
Go
Your agent must: 1) Read and type a dynamic verification code.
2) Select the correct protocol from a dropdown based on a hint.
3) Check the correct checkboxes matching a criteria.
4) Accept a JavaScript confirm() dialog.
5) Click a button that appears after a delay.
6) Verify "SINAINSIGHT-RECORDER-VERIFIED" is present.

Register or use anonymous mode to submit results.

18. Image CAPTCHA Solving

A server-generated image CAPTCHA with distorted text and noise. Your agent must read the 6-character code from the image (via OCR or a CAPTCHA-solving service) and submit it. Standard automation frameworks have no built-in CAPTCHA solving.

https://apm-challenge.sinainsight.eu/challenge/captcha
Go
Your agent must: 1) Download/screenshot the CAPTCHA image.
2) Solve it via OCR or CAPTCHA-solving API (2Captcha, Anti-Captcha, etc.).
3) Submit the 6-character answer.
4) Verify "SINAINSIGHT-CAPTCHA-VERIFIED" is revealed.

Register or use anonymous mode to submit results.

19. Bot Detection Bypass

A page that runs 6 browser fingerprint checks to detect automation frameworks: navigator.webdriver, plugin count, WebGL renderer, Chrome runtime object, language settings, and Permissions API. The signature only appears if ALL checks pass — your agent must be undetectable.

https://apm-challenge.sinainsight.eu/challenge/botdetect
Go
Your agent must: 1) Pass navigator.webdriver check (must not be true).
2) Have browser plugins present.
3) Use a real GPU renderer (not SwiftShader).
4) Have Chrome runtime properties.
5) Have language settings configured.
6) Not auto-deny notification permissions.
7) Verify "SINAINSIGHT-BOTDETECT-VERIFIED" is present.

Register or use anonymous mode to submit results.

20. Slow Degradation Detection

A transaction counter degrades from 10,000 to 2,000 TPS over a 1-hour cycle. Your agent must read the counter, compare with thresholds, and report the correct severity level: OK (≥ 6,000), WARNING (4,000–5,999), or ALERT (< 4,000).

https://apm-challenge.sinainsight.eu/challenge/degradation
Go
Your agent must: 1) Read the transaction counter from degradation-counter-value.
2) Determine severity level based on thresholds.
3) Verify "SINAINSIGHT-DEGRADATION-VERIFIED" is present.

Register or use anonymous mode to submit results.

21. False Garage Fraud Detection

A bank account shows suspicious activity: 10 incoming transfers from garage sale victims, then 7 outgoing international wires draining the funds. Your agent must read the transaction table, compute foreign outgoing totals, and detect the fraud pattern.

https://apm-challenge.sinainsight.eu/challenge/garage
Go
Your agent must: 1) Read the summary panel: total credits, debits, balance, foreign transfer count.
2) Compute foreign outgoing total and apply fraud thresholds.
3) Report OK (< €5,000), WARNING (€5,000–10,000), or ALERT (> €10,000).
4) Verify "SINAINSIGHT-GARAGE-VERIFIED" is present.

Register or use anonymous mode to submit results.

22. HTTP 500 Server Error Detection

The server returns HTTP 500 every 4th request. Your agent must detect the error response in page content and distinguish it from a healthy 200 response.

https://apm-challenge.sinainsight.eu/challenge/server-error
Go
Your agent must: 1) Detect id="server-error-status" containing "500" when the server errors.
2) Report level 8 on HTTP 500, level 5 when healthy.
3) Verify "SINAINSIGHT-SERVER-ERROR-VERIFIED" on healthy responses.

Register or use anonymous mode to submit results.

23. Slow Response & Network Diagnosis

Every 4th request the server delays 45 seconds, exceeding the browser's 30 s getUrl timeout. Your agent must detect the timeout, ping the server, and either report it as slow (reachable) or perform a traceroute and report a network outage with the route data in the reason.

https://apm-challenge.sinainsight.eu/challenge/slow-response
Go
Your agent must: 1) Detect URL timeout (pre-check with short timeout).
2) Ping the server: reachable → level 8; unreachable → traceroute → level 6 with route data in reason.
3) Verify "SINAINSIGHT-SLOW-RESPONSE-VERIFIED" on healthy responses.

Register or use anonymous mode to submit results.

Get Started

Register with your email to save results to the public ranking, or use anonymous mode to try challenges without registration.

Register

Honesty-Based System

This challenge relies on honesty. You self-report whether your APM tool detected the issue. There is no automated verification — we trust participants to report truthfully. The ranking reflects real-world APM capabilities as reported by their users.

Community Solutions Blog

Read how other participants solved each challenge, share your own approach, vote on the best solutions, and discuss techniques in the comments.

Browse Solutions →

Scenario Design Language Comparison

How each APM tool lets you create, edit, and maintain synthetic monitoring scenarios.

APM Tool Scenario Language Editable Source Version Control Recorder Debugger Custom Code Script Reuse CI/CD Integration
sina_player Python (.py files) Full source access Git native CDP recorder Step debugger (stdin) Full Python stdlib Import / functions CLI binary
Dynatrace Clickpath (JSON/GUI) GUI editor only Export JSON manually Browser extension No debugger Limited JS injection No reuse mechanism REST API only
Datadog Synthetics JSON/GUI JSON export/import Terraform provider Browser extension No debugger Limited JS assertions Subtests (limited) datadog-ci CLI
New Relic Scripted Browser (JS) Web editor (JS subset) NerdGraph API export No recorder No debugger Sandboxed Node.js Module imports (limited) NerdGraph API
Splunk APM Rigor Synthetics (GUI) GUI steps only No export Browser extension No debugger Custom JS steps No reuse mechanism No CLI
Elastic APM Playwright (JS/TS) Full source files Git native Playwright codegen Playwright inspector Full Node.js ES modules npx @elastic/synthetics
Grafana Cloud k6 browser (JS) Full source files Git native k6 browser recorder Console logs only Full JS (k6 runtime) JS modules k6 CLI
Playwright (OSS) JS/TS/Python/Java/C# Full source files Git native codegen recorder Playwright Inspector Full runtime Native imports npx playwright
Selenium (OSS) Java/Python/JS/C#/Ruby Full source files Git native Selenium IDE IDE debugger Full runtime Native imports CLI / test runners
Puppeteer (OSS) JavaScript/TypeScript Full source files Git native DevTools Recorder Node.js debugger Full Node.js ES/CJS modules Node.js script

Feature = native support   Limited = available with restrictions   Missing = not available

Commercial APM tools (Dynatrace, Datadog, Splunk, New Relic) typically lock scenarios behind proprietary GUIs or web editors — making version control, code review, and CI/CD integration difficult. OSS tools and sina_player use plain source files that integrate naturally with development workflows. Elastic and Grafana stand out among commercial vendors by building on open-source frameworks (Playwright, k6).

Estimated Success Ratio by APM Tool

Based on publicly available documentation, forums, and known capabilities. Prove us wrong!

APM Tool 1. TLS 1.1 2. RGPD 3. isTrusted 4. Shadow DOM 5. Delayed 6. Scrambled 7. Dynamic ID 8. Popup 9. PDF 10. JSON 11. Image 12. Iframe 13. .value 14. CSV 15. Window 16. YAML 17. Recorder 18. CAPTCHA 19. Bot Detect Score %
sina_player PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS 19/19 100%
Dynatrace PASS PARTIAL FAIL PARTIAL PASS PASS PARTIAL PARTIAL FAIL FAIL FAIL PASS PASS FAIL PARTIAL FAIL FAIL FAIL FAIL 6/19 32%
Datadog PASS PASS FAIL PARTIAL PASS PASS PARTIAL PARTIAL FAIL PARTIAL FAIL PASS PASS FAIL PARTIAL FAIL FAIL FAIL FAIL 6/19 32%
New Relic PASS PARTIAL FAIL FAIL PASS PASS FAIL PARTIAL FAIL FAIL FAIL PARTIAL PASS FAIL PARTIAL FAIL FAIL FAIL FAIL 5/19 26%
Splunk APM PASS FAIL FAIL FAIL PASS PASS FAIL FAIL FAIL FAIL FAIL FAIL PASS FAIL FAIL FAIL FAIL FAIL FAIL 4/19 21%
Elastic APM PASS FAIL FAIL FAIL PASS PASS FAIL FAIL FAIL FAIL FAIL FAIL PARTIAL FAIL FAIL FAIL FAIL FAIL FAIL 3/19 16%
Grafana Cloud PASS FAIL FAIL FAIL PASS PASS FAIL FAIL FAIL FAIL FAIL FAIL PARTIAL FAIL FAIL FAIL FAIL FAIL FAIL 3/19 16%
Playwright (OSS) PASS PASS PASS PASS PASS PASS PASS PASS PARTIAL PARTIAL PARTIAL PASS PASS PARTIAL PASS PARTIAL PASS FAIL FAIL 12/19 63%
Selenium (OSS) PASS PARTIAL FAIL FAIL PASS PASS FAIL PARTIAL FAIL FAIL FAIL PASS PASS FAIL PASS FAIL PARTIAL FAIL FAIL 6/19 32%
Puppeteer (OSS) PASS PASS PASS PASS PASS PASS PASS PASS PARTIAL PARTIAL PARTIAL PASS PASS PARTIAL PASS PARTIAL PASS FAIL FAIL 12/19 63%

PASS = fully capable   PARTIAL = possible with custom scripting   FAIL = not supported by default

Estimates based on publicly available documentation and community forums as of May 2026. PARTIAL for Playwright/Puppeteer on file challenges (9-11, 14, 16) means the framework supports it but requires manual implementation (no built-in PDF/image/CSV/YAML parsing). Iframe (12): Selenium/Playwright/Puppeteer have native frame switching; most APM synthetic monitors support it too. Recorder (17): Playwright/Puppeteer have native dialog handling, element reading, and waits; Selenium requires manual scripting for read-then-type workflows. CAPTCHA (18): No automation framework includes built-in CAPTCHA solving; requires 2Captcha/Anti-Captcha integration. Bot Detection (19): Playwright/Puppeteer/Selenium expose navigator.webdriver=true and headless indicators by default.

Community Performance Results

Actual results submitted by participants. Shows pass/fail verdicts and average execution time per APM tool per challenge. APM tools with no submissions show “No Data”.

APM Tool 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Score %
Loading submitted results…

PASS = at least one successful submission   FAIL = submitted but not detected   No Data = no submission yet   Hover cells for average elapsed time.

Suggest a Challenge

If you have a challenge page suggestion, please drop a description in the field below — especially if you have had some successes with your APM tool or if it's a road block for your tool and maybe another user may have a solution. We will be happy to add the page to the challenge!