Surge API

Welcome to the Surge API! You can use the API to programmatically create projects and run tasks. All you need is your API key, and funds in your account if you want to have tasks completed by our workforce.

Authentication

To authorize, use this code.

require 'httparty'

# For POST requests
HTTParty.post("{{API_ENDPOINT}}",
  basic_auth: { username: "{{YOUR_API_KEY}}" },
  body: {
    ... # Parameters in the POST request
  }
)

# For GET requests
HTTParty.get("{{API_ENDPOINT}}",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

# For POST requests
requests.post("{{API_ENDPOINT}}",
  auth = ("{{YOUR_API_KEY}}", ""),
  json = {
    ... # Parameters in the POST request
  }
)

# For GET requests
requests.get("{{API_ENDPOINT}}",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

# Or set the API key as an environment variable:
# export SURGE_API_KEY={{YOUR_API_KEY}}

# Note that the colon follows the API key and is not part of it.
curl {{API_ENDPOINT}} \
  -u {{YOUR_API_KEY}}:

Surge uses API keys to allow access to the API. You can find your API key in your profile.

Authentication is performed via HTTP Basic Auth. Your API key is your basic auth username, and you do not need to provide a password.

If you need to authenticate via bearer auth (e.g., for a cross-origin request), use -H "Authorization: {{YOUR_API_KEY}}:" instead of -u {{YOUR_API_KEY}}:.

Example

Let's walk through an example using Surge to categorize companies. If you'd like to run the example yourself, find your API key in your profile.

require 'httparty'

# First, we create a "categorize company" project.
response = HTTParty.post("https://app.surgehq.ai/api/projects",
  basic_auth: { username: "DJ34KFX24M2G3JRHGXNZVCPE" },
  body: {
    name: "Categorize this company",
    payment_per_response: 0.1,
    questions: [
      {
        "text": "What is this company's website?"
      },
      {
        "text": "What category does this company belong to?",
        "options": ["Tech", "Sports", "Gaming"]
      }
    ]
  }
)

# Extract the project ID.
project_id = JSON.parse(response.body)["id"]

# Post a task to the API.
HTTParty.post("https://app.surgehq.ai/api/projects/#{project_id}/tasks",
  basic_auth: { username: "DJ34KFX24M2G3JRHGXNZVCPE" },
  body: {
    "fields": {
      "website": "google.com"
    }
  }
)

# Retrieve the tasks and any responses.
response = HTTParty.get("https://app.surgehq.ai/api/projects/#{project_id}/tasks",
  basic_auth: { username: "DJ34KFX24M2G3JRHGXNZVCPE" }
)

JSON.parse(response.body).each do |task|
  puts task["responses"][0]["response_data"]
end

import requests

requests.post("https://app.surgehq.ai/api/projects",
  auth = ("{{YOUR_API_KEY}}", ""),
  json = {
    "name": "Categorize this company",
    "payment_per_response": 0.1,
    "questions": [
      {
        "text": "What is this company's website?"
      },
      {
        "text": "What category does this company belong to?",
        "options": ["Tech", "Sports", "Gaming"]
      }
    ]
  }
)

import surge
from surge.questions import FreeResponseQuestion, MultipleChoiceQuestion
surge.api_key = "{{YOUR_API_KEY}}"

# Create questions to add to the project
free_response_q = FreeResponseQuestion(
    text="What is this company's website?")

multiple_choice_q = MultipleChoiceQuestion(
    text="What category does this company belong to?",
    options=["Tech", "Sports", "Gaming"])

project = surge.Project.create(
    name="Categorize this company",
    payment_per_response=0.1,
    questions=[free_response_q, multiple_choice_q])

Projects

Create a project

Definition

POST https://app.surgehq.ai/api/projects

Example Request

curl https://app.surgehq.ai/api/projects \
  -u {{YOUR_API_KEY}}: \
  -d name="Categorize this company" \
  -d payment_per_response=0.1 \
  -d questions=... (can't do this in curl)

require 'httparty'

HTTParty.post("https://app.surgehq.ai/api/projects",
  basic_auth: { username: "{{YOUR_API_KEY}}" },
  body: {
    name: "Categorize this company",
    payment_per_response: 0.1,
    instructions: "You will be asked to categorize a company.",
    questions: [
      {
        "type": "free_response",
        "text": "What is this company's website?"
      },
      {
        "type": "multiple_choice",
        "text": "What category does this company belong to?",
        "options": ["Tech", "Sports", "Gaming"]
      },
      {
        "type": "checkbox",
        "text": "Check all the social media accounts this company has",
        "options": ["Facebook", "Twitter", "Pinterest", "Google+"]
      }
    ]
  }
)

import requests

requests.post("https://app.surgehq.ai/api/projects",
  auth = ("{{YOUR_API_KEY}}", ""),
  json = {
    "name": "Categorize this company",
    "payment_per_response": 0.1,
    "instructions": "You will be asked to categorize a company.",
    "questions": [
      {
        "type": "free_response",
        "text": "What is this company's website?"
      },
      {
        "type": "multiple_choice",
        "text": "What category does this company belong to?",
        "options": ["Tech", "Sports", "Gaming"]
      },
      {
        "type": "checkbox",
        "text": "Check all the social media accounts this company has",
        "options": ["Facebook", "Twitter", "Pinterest", "Google+"]
      }
    ]
  }
)

import surge
from surge.questions import (FreeResponseQuestion, MultipleChoiceQuestion,
                             CheckboxQuestion)
surge.api_key = "{{YOUR_API_KEY}}"

# Create questions to add to the project
free_response_q = FreeResponseQuestion(
    text="What is this company's website?")

multiple_choice_q = MultipleChoiceQuestion(
    text="What category does this company belong to?",
    options=["Tech", "Sports", "Gaming"])

checkbox_q = CheckboxQuestion(
    text="Check all the social media accounts this company has",
    options=["Facebook", "Twitter", "Pinterest", "Google+"])

project = surge.Project.create(
    name="Categorize this company",
    payment_per_response=0.1,
    instructions="You will be asked to categorize a company.",
    questions=[free_response_q, multiple_choice_q, checkbox_q])

Example Response

Project({
  "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
  "name": "Categorize this company",
  "num_tasks": 0,
  "num_tasks_completed": 0,
  "num_workers_per_task": 1,
  "payment_per_response": 0.1,
  "status": "unlaunched"
})

Creates a project. It is also possible to create a project through the web UI (instead of the API), and you will still be able to use the API to post tasks to the project.

Parameters

Parameter	Type	Description
name	string	Required. Name to give your project.
private_workforce	boolean	Optional (default false). If true, the project won't be shown to the Surge workforce and will only be available to workers who receive the link to your task. If false,
payment_per_response	float	Optional, only required for tasks being sent to the Surge workforce. Amount in dollars to pay workers for each response. You will need enough funds in your account to pay for all of the responses required by the project before launching it.
instructions	string	Optional. Instructions shown to workers describing how they should complete the task.
fields_template	string	Optional. A template describing how fields are shown to workers working on the task. For example, if `fields_template` is `<a href="{{url}}">{{company_name}}</a>`, then workers will be shown a link to the company.
questions	array	Required. An array of question objects describing the questions to be answered. Each question object contains a `text` field (the text of the question), an optional `options` array (for multiple choice and checkbox questions, listing the answer options), a `type` field (one of `free_response`, `multiple_choice`, or `checkbox`), and a `required` boolean field (describing whether or not workers must answer the question). See the Question Types section for a list of available question types.
num_workers_per_task	integer	Optional (default 1). Number of workers who work on each task.

Question Types

This table describes each of the available questions you can send to workers

Question Type	Description	Screenshot of worker view
Multiple Choice	The worker is presented with a multiple choice question and can only select one answer. SDK Object: `MultipleChoiceQuestion(text, options, required=True)`
Checkbox	The worker is presented with a series of options and can check as many as they want. SDK Object: `CheckboxQuestion(text, options)`
Free Response	The worker is presented with a text area input they can type into. SDK Object: `FreeResponseQuestion(text, required=True)`
Text Tagging (Named Entity Recognition)	The worker is presented text and a list of options. They can highlight spans of text to tag them. SDK Object: `TextTaggingQuestion(text, options)`

Retrieve a project

Definition

GET https://app.surgehq.ai/api/projects/{{PROJECT_ID}}

Example Request

require 'httparty'

HTTParty.get("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.get("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

curl https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8 \
  -u {{YOUR_API_KEY}}:

Example Response

Project({
  "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
  "name": "Categorize this company",
  "num_tasks": 1000,
  "num_tasks_completed": 239,
  "num_workers_per_task": 1,
  "payment_per_response": 0.1,
  "status": "unlaunched"
})

Retrieves a specific project you have created.

Parameter	Type	Description
id	string	Unique identifier for project.
created_at	DateTime	When the project was created.
fields_template	string	A template describing how fields are shown to workers working on the task. For example, if `fields_template` is `<a href="{{url}}">{{company_name}}</a>`, then workers will be shown a link to the company.
instructions	string	Instructions shown to workers describing how they should complete the task.
name	string	Name of the project.
link_to_work_on_task	string	URL for working on the task. If you are using the private workforce option, this is the URL you will want to send to your workforce.
num_tasks	integer	Number of tasks in the project.
num_tasks_completed	integer	Number of completed tasks.
num_workers_per_task	integer	How many workers work on each task (i.e., how many responses per task).
payment_per_response	float	How much a worker is paid (in US dollars) for an individual response.
questions	array	An array of question objects describing the questions to be answered. Each question object contains a `text` field (the text of the question), an optional `options` array (for multiple choice and checkbox questions, listing the answer options), a `type` field (one of `free_response`, `multiple_choice`, or `checkbox`), and a `required` boolean field (describing whether or not workers must answer the question). See the Question Types section for a list of available question types.
status	string	One of `in_progress`, `completed`, `canceled`, or `paused`.
interrater_agreement	Dictionary	A dictionary mapping each question to the its interrater agreement score.
avg_gold_standard_score	Float	The average correctness of the project's gold standard responses, ranging from 0-100. For gold standards set multiple choice questions, this is the percent of workers who answered correctly. For checkbox questions, this is the proportion of checkboxes where the worker correctly checked it or left it unchecked. For text tagging questions this is the average of a similarity score between the gold standard tags and worker tags.

List all projects

Definition

GET https://app.surgehq.ai/api/projects
GET https://app.surgehq.ai/api/projects?page=n

Example Request

require 'httparty'

HTTParty.get("https://app.surgehq.ai/api/projects?page=2",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.get("https://app.surgehq.ai/api/projects",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

projects = surge.Project.list(page=1)

curl https://app.surgehq.ai/api/projects \
  -u {{YOUR_API_KEY}}:

Example Response

[
  Project({
    "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
    "name": "Categorize this company",
    "num_tasks": 1000,
    "num_tasks_completed": 239,
    "num_workers_per_task": 1,
    "payment_per_response": 0.1,
    "status": "unlaunched"
  }),
  Project({
    "id": "81533541-0359-4d4b-a545-af38a2cb3e8c",
    "name": "Label product images",
    "num_tasks": 5000,
    "num_tasks_completed": 4315,
    "num_workers_per_task": 1,
    "payment_per_response": 0.25,
    "status": "in_progress"
  })
]

Get a list of all the projects you have created. Returns a list of Project objects.

Parameters

Parameter	Type	Description
page	integer	Projects are returned in descending `created_at` order. Each page contains a maximum of 25 projects. Pages start at 1. (optional, default 1)

Pause a project

Definition

POST https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/pause

Example Request

require 'httparty'

HTTParty.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/pause",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/pause",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

project = project.pause()

curl https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/pause \
  -u {{YOUR_API_KEY}}: \
  -X POST

Example Response

Project({
  "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
  "name": "Categorize this company",
  "num_tasks": 1000,
  "num_tasks_completed": 239,
  "num_workers_per_task": 1,
  "payment_per_response": 0.1,
  "statu)s": "paused"
}

Pauses a project. Tasks added to the project will not be worked on until you resume the project.

Resume a project

Definition

POST https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/resume

Example Request

require 'httparty'

HTTParty.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/resume",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/resume",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

project = project.resume()

curl https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/resume \
  -u {{YOUR_API_KEY}}: \
  -X POST

Example Response

Project({
  "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
  "name": "Categorize this company",
  "num_tasks": 1000,
  "num_tasks_completed": 239,
  "num_workers_per_task": 1,
  "payment_per_response": 0.1,
  "status": "in_progress"
})

Resumes a paused project.

Cancel a project

Definition

POST https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/cancel

Example Request

require 'httparty'

HTTParty.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/cancel",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/cancel",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

project = project.cancel()

curl https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/cancel \ 
  -u {{YOUR_API_KEY}}: \
  -X POST

Example Response

Project({
  "id": "17e323f1-f7e4-427c-a2d5-456743aba8",
  "name": "Categorize this company",
  "num_tasks": 1000,
  "num_tasks_completed": 239,
  "num_workers_per_task": 1,
  "payment_per_response": 0.1,
  "status": "canceled"
})

Cancels a project.

Tasks

The task object

Example Response

Task({
  "id": "38da6bc5-a644-41b9-a964-4678bc7375c6",
  "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
  "created_at": "2016-11-01T18:56:56.000Z",
  "is_complete": true,

  "fields": {
    "company": "Surge",
    "city": "San Francisco",
    "state": "CA"
  },

  "responses": [
    TaskResponse({
      "id": "1befb37b-8e4f-42b6-8a61-2c946ad4b5ce",
      "data": {
        "website": "https://app.surgehq.ai",
        "category": "Technology"
      },
      "time_spent_in_secs": 2,
      "completed_at": "2016-11-01T23:29:17.971Z",
      "worker_id": "P94T3ATCZX4X"
    })
  ]
})

Parameter	Type	Description
id	string	Unique identifier for the task.
created_at	DateTime	When the task was created.
project_id	string	ID of the project that this task belongs to.
is_complete	boolean	Whether or not this task has received the desired number of responses (equal to the project's `num_workers_per_task` field).
fields	dictionary	A dictionary of named fields that get shown (according to the project template) to workers when working on the task.
responses	array	An array of TaskResponses to the task. Each TaskResponse object contains an `id`, `time_spent_in_secs` field, `completed_at` field, `worker_id` field, and a `data` dictionary that maps questions to their answers.

Create a task

Definition

POST https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/tasks

Example Request

require 'httparty'

HTTParty.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/tasks",
  basic_auth: { username: "{{YOUR_API_KEY}}" },
  body: {
    "fields": {
      "company": "Surge",
      "city": "San Francisco",
      "state": "CA"
    }
  }
)

import requests

requests.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/tasks",
  auth = ("{{YOUR_API_KEY}}", ""),
  json = {
    "fields": {
      "company": "Surge",
      "city": "San Francisco",
      "state": "CA"
    }
  }
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

tasks = project.create_tasks([{
    "company": "Surge",
    "city": "San Francisco",
    "state": "CA"
}])

# You can also create Tasks in bulk via CSV file.
# The header of the CSV file must specify the fields that are used in your Tasks.
file_path = './companies_to_classify.csv'
tasks = project.create_tasks_from_csv(file_path)

curl https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/tasks \
  -u {{YOUR_API_KEY}}: \
  -d data=... (can't do this in curl)

Example Response

Task({
  "id": "38da6bc5-a644-41b9-a964-4678bc7375c6",
  "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
  "created_at": "2016-11-01T18:56:56.000Z",
  "is_complete": false,

  "fields": {
    "company": "Surge",
    "city": "San Francisco",
    "state": "CA"
  },

  "responses": []
})

Creates a task.

Whenever a task is created, the project is automatically launched if it is not already in progress.

Parameters

Parameter	Type	Description
data	dictionary	A dictionary of named fields that get shown (according to the project template) to workers when working on the task. (required)

List all tasks

Definition

GET https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/tasks
GET https://app.surgehq.ai/api/projects/{{PROJECT_ID}}/tasks?page=n

Example Request

require 'httparty'

HTTParty.get("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/tasks",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.post("https://app.surgehq.ai/api/projects/17e323f1-f7e4-427c-a2d5-456743aba8/tasks",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

project = surge.Project.retrieve("17e323f1-f7e4-427c-a2d5-456743aba8")

tasks = project.list_tasks(page=1)

curl https://app.surgehq.ai/api/projects/b31ede78-afdf-4938-b775-8813453c7fc5/tasks \
  -u {{YOUR_API_KEY}}: \
  ...

Example Response

[
  Task({
    "id": "38da6bc5-a644-41b9-a964-4678bc7375c6",
    "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
    "created_at": "2016-11-01T18:56:56.000Z",
    "is_complete": true,

    "fields": {
      "company": "Surge",
      "city": "San Francisco",
      "state": "CA"
    },

    "responses": [
      TaskResponse({
        "id": "1befb37b-8e4f-42b6-8a61-2c946ad4b5ce",
        "data": {
          "website": "https://app.surgehq.ai",
          "category": "Technology"
        },
        "time_spent_in_secs": 2,
        "completed_at": "2016-11-01T23:29:17.971Z",
        "worker_id": "P94T3ATCZX4X"
      })
    ]
  }),
  Task({
    "id": "dd66e80a-88b3-4c4f-9efc-2aca8eed73db",
    "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
    "created_at": "2016-11-01T23:23:26.016Z",
    "is_complete": true,

    "fields": {
      "company": "Google",
      "city": "Mountain View",
      "state": "CA"
    },

    "responses": [
      TaskResponse({
        "id": "cc9f4263-1d0f-4730-a97e-199851b6ade5",
        "data": {
          "website": "https://www.google.com",
          "category": "Technology"
        },
        "time_spent_in_secs": 3,
        "completed_at": "2016-11-01T23:22:19.124Z",
        "worker_id": "P94T3ATCZX4X"
      })
    ]
  })
]

Lists all tasks belonging to a project.

Query Parameters

Parameters

Parameter	Type	Description
page	integer	Tasks are returned in ascending `created_at` order. Each page contains a maximum of 25 tasks. Pages start at 1. (optional, default 1)

Retrieve a task

Definition

GET https://app.surgehq.ai/api/tasks/{{TASK_ID}}

Example Request

require 'httparty'

HTTParty.get("https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6",
  basic_auth: { username: "{{YOUR_API_KEY}}" }
)

import requests

requests.post("https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6",
  auth = ("{{YOUR_API_KEY}}", "")
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

task = surge.Task.retrieve(task_id = "eaa44510-c8f6-4480-b746-28a6c8defd4c")

curl https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6 \
  -u {{YOUR_API_KEY}}:

Example Response

Task({
  "id": "38da6bc5-a644-41b9-a964-4678bc7375c6",
  "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
  "created_at": "2016-11-01T18:56:56.000Z",
  "is_complete": true,

  "fields": {
    "company": "Surge",
    "city": "San Francisco",
    "state": "CA"
  },

  "responses": [
    TaskResponse({
      "id": "1befb37b-8e4f-42b6-8a61-2c946ad4b5ce",
      "data": {
        "website": "https://app.surgehq.ai",
        "category": "Technology"
      },
      "time_spent_in_secs": 2,
      "completed_at": "2016-11-01T23:29:17.971Z",
      "worker_id": "P94T3ATCZX4X"
    })
  ]
})

Retrieves a specific task you have created.

Set a gold standard answer

Definition

GET https://app.surgehq.ai/api/tasks/{{TASK_ID}}/gold-standards

Example Request

require 'httparty'

HTTParty.get("https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6/gold-standards",
  basic_auth: { username: "{{YOUR_API_KEY}}" },
    body: {
    "answers": [
      "https://surgehq.ai",
      'Tech',
      ['Twitter']
    ]
  }
)

import requests

requests.post("https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6/gold-standards",
  auth = ("{{YOUR_API_KEY}}", ""),
  json = {
    "answers": [
      "https://surgehq.ai",
      'Tech',
      ['Twitter']
    ]
  }
)

import surge
surge.api_key = "{{YOUR_API_KEY}}"

task = surge.Task.retrieve(task_id = "eaa44510-c8f6-4480-b746-28a6c8defd4c")
task.set_gold_standard(['https://surgehq.ai', 'Tech', ['Twitter']])

curl https://app.surgehq.ai/api/tasks/38da6bc5-a644-41b9-a964-4678bc7375c6/gold-standards \
  -u {{YOUR_API_KEY}}
  -d data=... (can't do this in curl)

Example Response

Task({
  "id": "38da6bc5-a644-41b9-a964-4678bc7375c6",
  "project_id": "b31ede78-afdf-4938-b775-8813453c7fc5",
  "created_at": "2016-11-01T18:56:56.000Z",
  "is_complete": false,

  "fields": {
    "company": "Surge",
    "city": "San Francisco",
    "state": "CA"
  },
  "gold_standard_data": "{'ner': {},'multiple_choice': {'b6337c97-6498-4c8c-a9ea-fcdc45782067': '3366ef4e-fe7f-4606-92bf-fba9a60a3d19'},'checkbox': {'f3feb6b0-daf9-42e3-9154-fc66fe10802d': ['44f050e2-e936-4241-8a1b-991bdb64a53e']},'free_response': {'85470cbd-8309-42ab-be70-500eb644736d': 'https://surgehq.ai'}}"
})

Gold standards are used to assess worker quality. After a gold standard answer is uploaded, every single worker will be given that piece of data so they can all be evaluated based on their performance. The average correctness across all gold standards in a project are in the gold_standard_score field, and you can see additional analytics on the site, including how each worker is performing on gold standards and aggregated stats indicating which gold standards are answered incorrectly most often.

Parameters

Parameter	Type	Description
answers	List[String] or None	A list of the ground truth answers for this task, one for each question in the project. If you don't want to set an answer for one of the questions, you can leave it blank by passing an empty string. For example, if you wanted to modify the example on the right to only set the company categorization multiple choice question as a gold standard then you could set answers to `['', 'Tech', '']`.
answers	boolean	This param indicates whether this task is a gold standard. You can toggle gold standards on or off without modifying the answers by setting this value to true or false and omitting the answers array.

Task Responses

The TaskResponse object

Example Response

TaskResponse({
  "id": "1befb37b-8e4f-42b6-8a61-2c946ad4b5ce",
  "data": {
    "website": "https://app.surgehq.ai",
    "category": "Technology"
  },
  "time_spent_in_secs": 2,
  "completed_at": "2016-11-01T23:29:17.971Z",
  "worker_id": "P94T3ATCZX4X"
})

Parameter	Type	Description
id	string	A unique identifier for the task response.
data	dictionary	A dictionary that maps each question to the worker's response.
time_spent_in_secs	int	Time (in seconds) that the worker spent to complete this task response.
completed_at	string	Timestamp of the response completion.
worker_id	string	ID of the worker that completed this task response.