Return all items from a Github resource from the API
Posted on Tue 07 July 2020 in dev-journal
If you want to get the list of repos (or any other resource) from Github, the items are paginated once there are more than 30. Here is an example of how you can use a recursive function to follow the pagination and return all items of a particular resource.
Given you have your github api token you can use the following to get a list of repos
base_url = 'https://api.github.com/user/repos?per_page=100'
headers = {'Authorization': 'token {}'.format(github_token)}
repos = parse_github_pagination(baseUrl, headers)
def parse_github_pagination(base_url, headers, github_item=None):
response = requests.get(base_url, headers=headers)
if response.status_code != 200:
github_item = []
else:
if github_item:
github_item.extend(response.json())
else:
github_item = response.json()
links = parse_link_headers(response.headers)
if 'next' in links:
return parse_github_pagination(links['next'], headers, github_item)
else:
return github_item
The base_url can be changed to get any other resource you need that is
paginated.
For example, here is how you could use the same code above to get all of branches from a repository:
base_url = "https://api.github.com/repos/mikeabrahamsen/posthog/branches?per_page=100"
headers = {'Authorization': 'token {}'.format(github_token)}
branches = parse_github_pagination(baseUrl, headers)