Extract object/array value from rest API output using unix and jq

Question

When I run below curl query which makes call to REST API, it gives me two record in terms of issues. I would like to get "key" (SEA-739), "status" (Open) and "assignee" (null or emailAddress). Also how can i get count of no. of issues retrieved via "expand": operations,versionedRepresentations,editmeta,changelog,renderedFields",

I tried curl | jq '.issues[0].id' and it gives me valid id but I couldn't loop through both issues and get mentioned 3 values for both? How can I count issue using shell and get required output for all issues one by one?

JSON output from jq:

root@server# curl -u username:password -X GET -H 'Content-Type: application/json' --insecure --silent 'https://myserver.com/rest/api/2/search?jql=project=SE&status=Closed%20order%20by%20key%20desc&maxResults=2' | jq

{
  "expand": "schema,names",
  "startAt": 0,
  "maxResults": 2,
  "total": 733,
  "issues": [
    {
      "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
      "id": "292697",
      "self": "https://testserver.com/rest/api/2/issue/292697",
      "key": "SEA-739",
      "fields": {
    "customfield_10190": null,
    "customfield_13580": null,
    "customfield_10192": null,
    "customfield_10072": null,
    "customfield_13582": null,
    "customfield_10194": null,
    "customfield_13581": [],
    "customfield_10074": null,
    "customfield_13980": [
      {
        "self": "https://testserver.com/rest/api/2/customFieldOption/15071",
        "value": "check user base",
        "id": "15071"
      }
    ],
    "customfield_10075": null,
    "customfield_11680": null,
    "customfield_13583": null,
    "customfield_10076": null,
    "customfield_10230": null,
    "fixVersions": [],
    "customfield_10231": null,
    "customfield_13189": null,
    "customfield_13981": null,
    "customfield_10232": null,
    "resolution": null,
    "customfield_10233": null,
    "customfield_10234": null,
    "customfield_14380": null,
    "lastViewed": null,
    "customfield_10580": null,
    "customfield_12880": null,
    "customfield_10581": null,
    "customfield_12882": null,
    "customfield_10220": null,
    "customfield_12881": null,
    "priority": {
      "self": "https://testserver.com/rest/api/2/priority/4",
      "iconUrl": "https://testserver.com/images/icons/priorities/minor.svg",
      "name": "P4",
      "id": "4"
    },
    "customfield_10221": null,
    "customfield_10980": null,
    "customfield_10101": {
      "self": "https://testserver.com/rest/api/2/customFieldOption/10377",
      "value": "NO",
      "id": "10377"
    },
    "customfield_10222": null,
    "customfield_12883": "0|i16rjz:",
    "labels": [],
    "customfield_13208": null,
    "aggregatetimeoriginalestimate": null,
    "timeestimate": null,
    "versions": [],
    "issuelinks": [],
    "assignee": null,
    "status": {
      "self": "https://testserver.com/rest/api/2/status/1",
      "description": "The issue is open and ready for the assignee to start work on it.",
      "iconUrl": "https://testserver.com/images/icons/statuses/open.png",
      "name": "Open",
      "id": "1",
      "statusCategory": {
        "self": "https://testserver.com/rest/api/2/statuscategory/2",
        "id": 2,
        "key": "new",
        "colorName": "blue-gray",
        "name": "To Do"
      }
    },
    "components": [],
    "customfield_13281": null,
    "customfield_13280": null,
    "customfield_13283": null,
    "customfield_13282": null,
    "customfield_13680": null,
    "customfield_11780": "9223372036854775807",
    "customfield_11782": null,
    "customfield_13201": null,
    "customfield_11781": null,
    "customfield_10210": null,
    "customfield_13203": null,
    "customfield_13202": null,
    "customfield_13204": null,
    "aggregatetimeestimate": null,
    "customfield_14480": "com.atlassian.servicedesk.plugins.approvals.internal.customfield.ApprovalsCFValue@7735d8eb",
    "creator": {
      "self": "https://testserver.com/rest/api/2/user?username=user1",
      "name": "user1",
      "key": "user1",
      "emailAddress": "[email protected]",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/useravatar?avatarId=10312",
        "24x24": "https://testserver.com/secure/useravatar?size=small&avatarId=10312",
        "16x16": "https://testserver.com/secure/useravatar?size=xsmall&avatarId=10312",
        "32x32": "https://testserver.com/secure/useravatar?size=medium&avatarId=10312"
      },
      "displayName": "User1",
      "active": true,
      "timeZone": "Asia/Kolkata"
    },
    "customfield_10280": null,
    "subtasks": [],
    "customfield_10160": "org.hakanai.jira.plugins.StatusColor@50",
    "customfield_12580": null,
    "reporter": {
      "self": "https://testserver.com/rest/api/2/user?username=user1",
      "name": "user1",
      "key": "user1",
      "emailAddress": "[email protected]",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/useravatar?avatarId=10312",
        "24x24": "https://testserver.com/secure/useravatar?size=small&avatarId=10312",
        "16x16": "https://testserver.com/secure/useravatar?size=xsmall&avatarId=10312",
        "32x32": "https://testserver.com/secure/useravatar?size=medium&avatarId=10312"
      },
      "displayName": "User1",
      "active": true,
      "timeZone": "Asia/Kolkata"
    },
    "customfield_12980": null,
    "aggregateprogress": {
      "progress": 0,
      "total": 0
    },
    "customfield_10200": null,
    "progress": {
      "progress": 0,
      "total": 0
    },
    "customfield_14080": null,
    "issuetype": {
      "self": "https://testserver.com/rest/api/2/issuetype/25",
      "id": "25",
      "description": "test account checking",
      "iconUrl": "https://testserver.com/download/resources/com.thed.zephyr.je/images/icons/ico_zephyr_issuetype.png",
      "name": "Test",
      "subtask": false
    },
    "customfield_11080": null,
    "customfield_14191": null,
    "customfield_11081": null,
    "customfield_11082": null,
    "customfield_11083": null,
    "customfield_10270": null,
    "customfield_11084": null,
    "customfield_13780": null,
    "timespent": null,
    "customfield_10150": 99999,
    "customfield_11480": null,
    "customfield_13782": null,
    "customfield_10152": null,
    "customfield_13781": null,
    "project": {
      "self": "https://testserver.com/rest/api/2/project/10162",
      "id": "10162",
      "key": "SEA",
      "name": "Test Automation",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/projectavatar?pid=10162&avatarId=10032",
        "24x24": "https://testserver.com/secure/projectavatar?size=small&pid=10162&avatarId=10032",
        "16x16": "https://testserver.com/secure/projectavatar?size=xsmall&pid=10162&avatarId=10032",
        "32x32": "https://testserver.com/secure/projectavatar?size=medium&pid=10162&avatarId=10032"
      },
      "projectCategory": {
        "self": "https://testserver.com/rest/api/2/projectCategory/10020",
        "id": "10020",
        "description": "All IT related tasks and projects",
        "name": "Corporate IT"
      }
    },
    "customfield_11880": null,
    "aggregatetimespent": null,
    "resolutiondate": null,
    "workratio": -1,
    "customfield_14190": null,
    "customfield_14182": null,
    "customfield_14183": null,
    "watches": {
      "self": "https://testserver.com/rest/api/2/issue/SEA-739/watchers",
      "watchCount": 1,
      "isWatching": false
    },
    "customfield_14181": null,
    "customfield_14186": null,
    "customfield_10380": null,
    "customfield_12680": null,
    "created": "2016-10-29T16:17:57.000-0400",
    "customfield_10260": null,
    "customfield_14185": null,
    "customfield_10140": null,
    "customfield_10261": null,
    "customfield_10020": {
      "self": "https://testserver.com/rest/api/2/customFieldOption/10380",
      "value": "S3",
      "id": "10380"
    },
    "customfield_10262": null,
    "customfield_10780": null,
    "updated": "2016-10-29T16:17:57.000-0400",
    "customfield_13081": null,
    "customfield_13080": null,
    "customfield_13083": "To be filled in by Engineering",
    "customfield_11180": null,
    "customfield_13082": null,
    "customfield_11181": null,
    "customfield_13085": null,
    "customfield_11182": null,
    "customfield_13084": null,
    "customfield_11580": null,
    "customfield_11183": null,
    "customfield_13087": null,
    "customfield_13086": null,
    "timeoriginalestimate": null,
    "customfield_13881": null,
    "customfield_10130": null,
    "description": "Please ignore test bug..",
    "customfield_13880": null,
    "customfield_10011": null,
    "customfield_10253": null,
    "customfield_10012": null,
    "customfield_10007": null,
    "customfield_10008": null,
    "customfield_10009": null,
    "summary": "Please ignore test bug..",
    "customfield_13191": null,
    "customfield_13190": null,
    "customfield_14282": null,
    "customfield_13193": null,
    "customfield_13192": null,
    "customfield_14280": null,
    "customfield_13195": null,
    "customfield_14285": [
      {
        "self": "https://testserver.com/rest/api/2/customFieldOption/15507",
        "value": " YES",
        "id": "15507"
      }
    ],
    "customfield_13194": null,
    "customfield_14283": null,
    "customfield_10480": null,
    "customfield_13196": null,
    "customfield_14284": null,
    "customfield_10085": null,
    "customfield_12780": null,
    "customfield_10120": null,
    "customfield_10241": null,
    "customfield_10088": null,
    "customfield_10121": null,
    "customfield_10000": null,
    "customfield_10242": null,
    "customfield_10880": null,
    "customfield_10089": null,
    "customfield_10122": null,
    "customfield_10243": null,
    "customfield_10002": null,
    "customfield_10244": null,
    "customfield_10486": null,
    "customfield_10124": null,
    "customfield_10003": null,
    "customfield_10245": null,
    "customfield_10487": null,
    "customfield_10488": null,
    "customfield_10236": null,
    "customfield_10237": null,
    "environment": null,
    "duedate": null
      }
    },

    {
      "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
      "id": "273625",
      "self": "https://testserver.com/rest/api/2/issue/273625",
      "key": "SEA-738",
      "fields": {
    "customfield_10190": null,
    "customfield_13580": null,
    "customfield_10192": null,
    "customfield_10072": null,
    "customfield_10193": null,
    "customfield_13582": null,
    "customfield_10194": null,
    "customfield_13581": [],
    "customfield_10074": null,
    "customfield_13980": null,
    "customfield_10075": null,
    "customfield_11680": null,
    "customfield_13583": null,
    "customfield_10076": null,
    "customfield_10230": null,
    "fixVersions": [],
    "customfield_10231": null,
    "customfield_13189": null,
    "customfield_13981": null,
    "customfield_10232": null,
    "customfield_11685": null,
    "resolution": {
      "self": "https://testserver.com/rest/api/2/resolution/1",
      "id": "1",
      "description": "A fix for this issue is checked into the tree and tested.",
      "name": "Fixed"
    },
    "customfield_10233": null,
    "customfield_10234": null,
    "customfield_14380": null,
    "lastViewed": null,
    "customfield_10580": null,
    "customfield_12880": null,
    "customfield_10581": null,
    "customfield_12882": null,
    "customfield_10220": null,
    "customfield_12881": null,
    "priority": {
      "self": "https://testserver.com/rest/api/2/priority/3",
      "iconUrl": "https://testserver.com/images/icons/priorities/major.svg",
      "name": "P3",
      "id": "3"
    },
    "customfield_10221": null,
    "customfield_10980": null,
    "customfield_10101": {
      "self": "https://testserver.com/rest/api/2/customFieldOption/10377",
      "value": "NO",
      "id": "10377"
    },
    "customfield_10222": null,
    "customfield_12883": "0|i13y67:",
    "labels": [],
    "customfield_13208": null,
    "aggregatetimeoriginalestimate": null,
    "timeestimate": null,
    "versions": [],
    "issuelinks": [],
    "assignee": {
      "self": "https://testserver.com/rest/api/2/user?username=user2",
      "name": "user2",
      "key": "user2",
      "emailAddress": "[email protected]",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/useravatar?avatarId=10312",
        "24x24": "https://testserver.com/secure/useravatar?size=small&avatarId=10312",
        "16x16": "https://testserver.com/secure/useravatar?size=xsmall&avatarId=10312",
        "32x32": "https://testserver.com/secure/useravatar?size=medium&avatarId=10312"
      },
      "displayName": "Russell Pratt",
      "active": true,
      "timeZone": "EST5EDT"
    },
    "status": {
      "self": "https://testserver.com/rest/api/2/status/5",
      "description": "A resolution has been taken and is awaiting verification by reporter. From here issues are either reopened or are closed.",
      "iconUrl": "https://testserver.com/images/icons/statuses/resolved.png",
      "name": "Resolved",
      "id": "5",
      "statusCategory": {
        "self": "https://testserver.com/rest/api/2/statuscategory/3",
        "id": 3,
        "key": "done",
        "colorName": "green",
        "name": "Done"
      }
    },
    "components": [],
    "customfield_13281": null,
    "customfield_13280": null,
    "customfield_13283": null,
    "customfield_13282": null,
    "customfield_13680": null,
    "customfield_11780": "9223372036854775807",
    "customfield_11782": null,
    "customfield_13201": null,
    "customfield_11781": null,
    "customfield_10210": null,
    "customfield_13203": null,
    "customfield_13202": null,
    "customfield_13204": null,
    "aggregatetimeestimate": null,
    "customfield_14480": "com.atlassian.servicedesk.plugins.approvals.internal.customfield.ApprovalsCFValue@c4b41a5",
    "creator": {
      "self": "https://testserver.com/rest/api/2/user?username=userads",
      "name": "userads",
      "key": "userads",
      "emailAddress": "[email protected]",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/useravatar?ownerId=userads&avatarId=15412",
        "24x24": "https://testserver.com/secure/useravatar?size=small&ownerId=userads&avatarId=15412",
        "16x16": "https://testserver.com/secure/useravatar?size=xsmall&ownerId=userads&avatarId=15412",
        "32x32": "https://testserver.com/secure/useravatar?size=medium&ownerId=userads&avatarId=15412"
      },
      "displayName": "User Ads",
      "active": true,
      "timeZone": "EST5EDT"
    },
    "customfield_10280": null,
    "subtasks": [],
    "customfield_10160": "org.hakanai.jira.plugins.StatusColor@54",
    "customfield_12580": null,
    "reporter": {
      "self": "https://testserver.com/rest/api/2/user?username=userads",
      "name": "userads",
      "key": "userads",
      "emailAddress": "[email protected]",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/useravatar?ownerId=userads&avatarId=15412",
        "24x24": "https://testserver.com/secure/useravatar?size=small&ownerId=userads&avatarId=15412",
        "16x16": "https://testserver.com/secure/useravatar?size=xsmall&ownerId=userads&avatarId=15412",
        "32x32": "https://testserver.com/secure/useravatar?size=medium&ownerId=userads&avatarId=15412"
      },
      "displayName": "User Ads",
      "active": true,
      "timeZone": "EST5EDT"
    },
    "customfield_12980": null,
    "aggregateprogress": {
      "progress": 0,
      "total": 0
    },
    "customfield_10200": null,
    "progress": {
      "progress": 0,
      "total": 0
    },
    "customfield_14080": null,
    "issuetype": {
      "self": "https://testserver.com/rest/api/2/issuetype/3",
      "id": "3",
      "description": "A task that needs to be done.",
      "iconUrl": "https://testserver.com/secure/viewavatar?size=xsmall&avatarId=14808&avatarType=issuetype",
      "name": "Task",
      "subtask": false,
      "avatarId": 14808
    },
    "customfield_11080": null,
    "customfield_14191": null,
    "customfield_11081": null,
    "customfield_11082": null,
    "customfield_11083": null,
    "customfield_10270": null,
    "customfield_11084": null,
    "customfield_13780": null,
    "timespent": null,
    "customfield_10150": null,
    "customfield_11480": null,
    "customfield_13782": null,
    "customfield_10152": null,
    "customfield_13781": null,
    "project": {
      "self": "https://testserver.com/rest/api/2/project/10162",
      "id": "10162",
      "key": "SEA",
      "name": "Test Automation",
      "avatarUrls": {
        "48x48": "https://testserver.com/secure/projectavatar?pid=10162&avatarId=10032",
        "24x24": "https://testserver.com/secure/projectavatar?size=small&pid=10162&avatarId=10032",
        "16x16": "https://testserver.com/secure/projectavatar?size=xsmall&pid=10162&avatarId=10032",
        "32x32": "https://testserver.com/secure/projectavatar?size=medium&pid=10162&avatarId=10032"
      },
      "projectCategory": {
        "self": "https://testserver.com/rest/api/2/projectCategory/10020",
        "id": "10020",
        "description": "All IT related tasks and projects",
        "name": "Corporate IT"
      }
    },
    "customfield_11880": null,
    "aggregatetimespent": null,
    "resolutiondate": "2016-04-21T09:59:34.000-0400",
    "workratio": -1,
    "customfield_14190": null,
    "customfield_14182": null,
    "customfield_14183": null,
    "watches": {
      "self": "https://testserver.com/rest/api/2/issue/SEA-738/watchers",
      "watchCount": 2,
      "isWatching": false
    },
    "customfield_14181": null,
    "customfield_14186": null,
    "customfield_10380": null,
    "customfield_12680": null,
    "created": "2016-04-07T13:11:56.000-0400",
    "customfield_10260": null,
    "customfield_14185": null,
    "customfield_10140": null,
    "customfield_10261": null,
    "customfield_10020": {
      "self": "https://testserver.com/rest/api/2/customFieldOption/10380",
      "value": "S3",
      "id": "10380"
    },
    "customfield_10262": null,
    "customfield_10780": null,
    "updated": "2016-04-21T10:27:28.000-0400",
    "customfield_13081": null,
    "customfield_13080": null,
    "customfield_13083": "To be filled in by Engineering",
    "customfield_11180": null,
    "customfield_13082": null,
    "customfield_11181": null,
    "customfield_13085": null,
    "customfield_11182": null,
    "customfield_13084": null,
    "customfield_11580": null,
    "customfield_11183": null,
    "customfield_13087": null,
    "customfield_13086": null,
    "timeoriginalestimate": null,
    "customfield_13881": null,
    "customfield_10130": null,
    "description": "We need to update CTL logins for Cuke and RC.Currently the way the automation is setup, we can only use one user per environment, so please pick one from below for UAT and a different one for Prod.\r\n\r\n1. uat_centurylinknet:\r\n  :username: [email protected]\r\n  :password: P@ssw0rd!\r\n2. uat_embarqmailcom:\r\n  :username: [email protected]\r\n  :password: P@ssw0rd!\r\n3. uat_qcom:\r\n  :username: [email protected]\r\n  :password: P@ssw0rd!\r\n4. prod_centurylinknet:\r\n  :username: [email protected]\r\n  :password: test03\r\n5. prod_embarqmailcom:\r\n  :username: [email protected]\r\n  :password: test25\r\n6. prod_qcom:\r\n  :username: [email protected]\r\n  :password: test01",
    "customfield_13880": null,
    "customfield_11980": null,
    "customfield_10011": null,
    "customfield_10253": null,
    "customfield_10012": null,
    "customfield_10007": null,
    "customfield_10008": null,
    "customfield_10009": null,
    "summary": "Update CTL Logins",
    "customfield_13191": null,
    "customfield_13190": null,
    "customfield_14282": null,
    "customfield_13193": null,
    "customfield_13192": null,
    "customfield_14280": null,
    "customfield_13195": null,
    "customfield_14285": null,
    "customfield_13194": null,
    "customfield_14283": null,
    "customfield_10084": null,
    "customfield_10480": null,
    "customfield_13196": null,
    "customfield_14284": null,
    "customfield_10085": null,
    "customfield_12780": null,
    "customfield_10120": null,
    "customfield_10241": null,
    "customfield_10088": null,
    "customfield_10121": null,
    "customfield_10000": [
      {
        "self": "https://testserver.com/rest/api/2/customFieldOption/10009",
        "value": "CenturyLink",
        "id": "10009"
      }
    ],
    "customfield_10242": null,
    "customfield_10880": null,
    "customfield_10089": null,
    "customfield_10122": null,
    "customfield_10243": null,
    "customfield_10002": null,
    "customfield_10244": null,
    "customfield_10486": null,
    "customfield_10124": null,
    "customfield_10003": null,
    "customfield_10245": null,
    "customfield_10487": null,
    "customfield_10488": null,
    "customfield_10236": null,
    "customfield_10237": null,
    "environment": null,
    "duedate": null
      }
    }
  ]
}

peak · Accepted Answer · 2016-11-02T14:10:43.333

1

You don't give any details about the expected form of the output, but the following jq filter produces the information you seem to want (but some of your requirements are not too clear to me):

(.issues[] | {key, status: .fields.status.name, assignee: .fields.assignee.emailAddress}),
(.issues|length)

Output:

 {  
  "key": "SEA-739",
  "status": "Open",
  "assignee": null
}
{
  "key": "SEA-738",
  "status": "Resolved",
  "assignee": "[email protected]"
}
2

Postscript

If you want a one-liner that will produce the values as a stream, without quotation marks, consider:

$ jq -r '.issues | ((.[] | (.key, .fields.status.name, .fields.assignee.emailAddress)),length)'

Output:

SEA-739
Open
null
SEA-738
Resolved
[email protected]
2

edited Nov 02 '16 at 14:10

answered Oct 30 '16 at 14:54

peak

105,803
17
152
177

1

Maybe you aligned `status` and `assignee` on the same level as `expand`, which it is - inside `issues[].fields`, but those two fields don't exist directly under `issues[]`, (but `expand` does`) – hmedia1 Nov 01 '16 at 07:47
1

@hmedia1 - The answer has been tweaked accordingly. – peak Nov 01 '16 at 14:45
This is really what i needed and I'm very near to accept this as an answer. As mentioned, now i need to key, status and assignee one by one. – Jitesh Sojitra Nov 02 '16 at 06:29
@peak, one more time I need your help in http://stackoverflow.com/questions/40396445/get-customized-output-from-rest-api-output-using-jq-linux – Jitesh Sojitra Nov 03 '16 at 08:02

score 1 · Answer 2 · answered Nov 01 '16 at 07:39

I would like to get "key" (SEA-739), "status" (Open) and "assignee" (null or emailAddress).

.... count of no. of issues retrieved via "expand": operations,versionedRepresentations,editmeta,changelog,renderedFields",

This:

jq '.issues[] | { key, status:.fields.status.name, assignee: .fields.assignee.emailAddress, "total issues":{expand,count:( .expand | split(",") | length ) } }'

Will result;

{
  "key": "SEA-739",
  "status": "Open",
  "assignee": null,
  "total issues": {
    "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
    "count": 5
  }
}
{
  "key": "SEA-738",
  "status": "Resolved",
  "assignee": "[email protected]",
  "total issues": {
    "expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
    "count": 5
  }
}

If that's what you're after. It's not entirely clear as you have some non-unique keys being used in different places, and you're explicit requirements to count the expand properties don't really make a lot of sense to me, but your requirements seemed well explained.

one more time need your help in http://stackoverflow.com/questions/40396445/get-customized-output-from-rest-api-output-using-jq-linux — Jitesh Sojitra, Nov 03 '16 at 08:01

Extract object/array value from rest API output using unix and jq

2 Answers2

Postscript

Linked