Return all items from a Github resource from the API

Posted on Tue 07 July 2020 in dev-journal

If you want to get the list of repos (or any other resource) from Github, the items are paginated once there are more than 30. Here is an example of how you can use a recursive function to follow the pagination and return all items of a particular resource.

Given you have your github api token you can use the following to get a list of repos

base_url = ''
headers = {'Authorization': 'token {}'.format(github_token)}
repos = parse_github_pagination(baseUrl, headers)

def parse_github_pagination(base_url, headers, github_item=None):
    response = requests.get(base_url, headers=headers)
    if response.status_code != 200:
        github_item = []
        if github_item:
            github_item = response.json()
        links = parse_link_headers(response.headers)
    if 'next' in links:
        return parse_github_pagination(links['next'], headers, github_item)
        return github_item

The base_url can be changed to get any other resource you need that is paginated.

For example, here is how you could use the same code above to get all of branches from a repository:

base_url = ""
headers = {'Authorization': 'token {}'.format(github_token)}
branches = parse_github_pagination(baseUrl, headers)