nexqdata/ docs

Jobs

Follow a job and the runs it created, and get a job's data.

A job is one start of an agent that can create one or more runs — for example, a large search split across sessions. You get one from start-job — the recommended way to start an agent — and group and pipeline runs create one job per agent they start.

Knowing when a job is done

There's no call that waits for a job. Start it, keep its agentJobHistoryId, and check on it:

curl "$NEXQDATA_API/api/scraper-agents/jobs?jobId=$JOB_ID" \
  -H "X-Api-Key: $NEXQDATA_API_KEY"
200 OK — abridged
{
  "success": true,
  "data": {
    "items": [
      {
        "agentJobHistoryId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
        "agentName": "Court records · TX",
        "status": 1,
        "statusName": "In Progress",
        "isTerminal": false,
        "completedRunCount": 0,
        "totalRunCount": 1,
        "totalDataCount": 0,
        "errorMessage": null
      }
    ],
    "nextCursor": null
  }
}

The job is finished when isTerminal is true. Then status says how it went:

statusMeaning
0Never run.
1In progress.
2In progress, but at least one of its runs has failed.
3Succeeded — its data is ready to download.
4, 5Failed. errorMessage and the job's runs say why.

Branch on the numeric status, not on statusName — the name is display text. Check every 5–15 seconds while a job is in progress; reading status isn't rate-limited, but there's nothing to gain from polling faster than the job can progress.

List jobs

GET/api/scraper-agents/jobs

Jobs across every agent you can access, newest first, cursor-paged.

Query parameterDefaultDescription
stateallactive, succeeded, or failed. Unlike run history, an unknown value is ignored rather than rejected.
jobIdReturn just this job.
searchMatches the agent name or id, the session, or the error message.
from, toOnly jobs in this date range.
pageSize25Up to 100.
cursorThe nextCursor from the previous page.

For one agent's jobs, use:

GET/api/scraper-agents/{id}/jobs

Uses the agent's id (GUID), and takes pageSize and cursor.

Useful fields

FieldMeaning
agentJobHistoryIdThe job id — the agentJobHistoryId that start-job returned.
status, statusName, isTerminalThe job's status code, its display name, and whether it has finished.
completedRunCount, totalRunCountHow many of its runs have finished, out of how many.
totalDataCount, totalErrorsData items collected and errors hit, across all its runs.
inputParametersThe inputs the job started with.
triggerSource, scheduleIdManual or Schedule, and which schedule, if any.
searchGroupRunId, pipelineRunIdThe group or pipeline run the job belongs to, if any.
errorMessageWhy the job failed, when it did.

To list jobs by outcome, filter with state (failed covers both failure codes) rather than matching on statusName.

A job's runs

GET/api/scraper-agents/jobs/{jobId}/runs

Every run the job created, with the same fields as run history. While any of the job's runs are unfinished, it re-reads them from the execution engine and updates the job's totals before answering — so it's also the way to refresh a job that looks stale. There's no GET for a single job — use this, or GET /api/scraper-agents/jobs?jobId={jobId}.

A job's data

GET/api/scraper-agents/jobs/{jobId}/export

Downloads everything the job collected. For a job with several runs, that's one combined zip.

GET/api/scraper-agents/jobs/{jobId}/dataset

Returns the job's datasetId, to preview or query the data before downloading it. It fails the same way a download does — see When data isn't available.

Counts

Endpointdata
GET /api/scraper-agents/jobs/active-countJobs still in progress, as a number.
GET /api/scraper-agents/jobs/status-countsJobs by status: all, demo, inProgress, inProgressWithFailure, succeeded, failure, failed.

On this page