API design

simple tasks api:

POST /v1/tasks/simple/

{

"command" : "python x.py --p1"

"s3_script_path": {"x.py":"www.s3.com/bucket-name/user_name/x.py"}

"max_retries": 5

"schedule-time": "YYYY-MM-DD Hour: Min: TimeZone"

"schedule":"ASAP"

"priority": 20

"priority-level": "global"

"idempotency_key":"sfafa324324"

}

returns: 202 success

processes the request asynchronously.

either schedule-time or schedule

priority-level can be local or global

400 if command, script, schedule time or schedule is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.

401 for unauthentic

403 for unauthorzied

500 for internal server error

recurring tasks api:

POST /v1/tasks/recurring/

input=

{

"command" : "python x.py --p1"

"s3_script_path": {"x.py":"www.s3.com/bucket-name/user_name/x.py"}

"max_retries": 5

"schedule-time": "YYYY-MM-DD Hour: Min: TimeZone"

"interval": "YYYY-MM-DD Hour: Min"

"schedule":"ASAP"

"priority": 20

"priority-level": "global",

"idempotency_key":"sfafa324324"

}

returns: 202 success

{

taskid:"23421"

}

processes the request asynchronously.

either schedule-time or schedule

priority-level can be local or global

400 if command, script, schedule time or schedule is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.will be added to base schedule time each time and scheduled after that much time.

401 for unauthentic

403 for unauthorzied

500 for internal server error

DAG based Jobs:

Create new dag:

POST /v1/tasks/dag/

{

"max_retries": 5

"priority": 20

"priority-level": "global",

"idempotency_key":"sfafa324324"

}

returns: 200 success

priority-level can be local or global

max retry is 1 by default

400 if command or script is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.

401 for unauthentic

403 for unauthorzied

500 for internal server error

This following api just adds job:

POST /v1/tasks/dag/

{

"command" : "python x.py --p1"

"s3_script_path": {"x.py":"www.s3.com/bucket-name/user_name/x.py"}

"max_retries": 5

"priority": 20

"priority-level": "global"

"dag_id": "123",

\"idempotency_key":"sfafa324324"

}

returns: 200 success

priority-level can be local or global

max retry is 1 by default

400 if command or script or dag_id is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.

401 for unauthentic

403 for unauthorized

500 for internal server error

This following api just to start

POST /v1/tasks/dag/

{

"command" : "start"

"dag_id": "1234"

"max_retries": 5

"priority": 20

"priority-level": "global",

"idempotency_key":"sfafa324324"

}

returns: 202 aceepted

max retry is 1 by default

400 if command or script or dag_id is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.

401 for unauthentic

403 for unauthorized

500 for internal server error

This is for making dependency:

POST /v1/tasks/dag/dependency/

{

"dependent_job_id" : "132",

"dependeee_job_id" : "123",

"dag_id" : "1345325"

"idempotency_key":"sfafa324324"

}

returns: 200 success

{

"id""

}

this returns DAG id

max retry is 1 by default

400 if command or script is not mentioned

if priority is not mentioned that it will be considered as 1 by default

and priority level will be local by default.

401 for unauthentic

403 for unauthorized

500 for internal server error

GET /v1/tasks/{id}

returns: 200

{

status:"running"

retries:"2"

}

status can be scheduled, running, completed, failed, retried

retries is number of retries already tried

paginated api for listing jobs:

GET /v1/tasks/?query=page_num=2&pagesize=10

returns: 200

{

{

job_id:"234"

status:"running"

retries:"2"

},

{

job_id:"2324"

status:"completed"

retries:"2"

}....

}

All the APIs above will have JWT in their headers for user authentication and authorization .

For all the above APIs idempotency key is mandatory else it will throw 400 bad request.

we can set limit on what will be number of times per minute we

Submissions

Displaying Solutions with Scores Ranging from 1 to 10