Skip to content

Fix pagination with listing self-hosted runners #202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

HenryNguyen5
Copy link
Contributor

@HenryNguyen5 HenryNguyen5 commented Sep 11, 2020

Fixes #227

@npalm npalm self-requested a review September 14, 2020 19:12
@npalm
Copy link
Member

npalm commented Sep 14, 2020

Thanks for the PR will check asap.

Copy link
Member

@npalm npalm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All works, one comment that I would like to discuss.

return DEFAULT_REGISTERED_RUNNERS;
});
mockOctokit.actions.listSelfHostedRunnersForRepo.mockImplementation(() => {
mockOctokit.paginate.mockImplementation(() => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if we can find a better way to mock the specific function, this will mock all pagnate calls. For now no issue but create potential problems in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is possible to add a check to tests, something like:

expect(mockOcktokit.paginate).toBeCalledWith(mockOctokit.actions.listSelfHostedRunnersForOrg, expect.any());

Haven't tested if it works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could scope the mocking of the implementation to each individual test, so that it is restored inbetween each test. Does that work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed it's already been scoped to scaleDown, I think it's fine for now unless you're uncomfortable with the current test implementation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The risk is when we add pagination to more github call they all now handled by the same mock, all calls to mockOctokit.paginate will gets the same mocked response. Wil works as long we not paginate any other call. If would prefer if we can come with a fix. Otherwise I am fine with creating an issue and keep it for the moment. @gertjanmaas can you have a look again?

@HenryNguyen5
Copy link
Contributor Author

Been testing this PR on our own org, I've discovered that the lambda ends up timing out because if a large amount of runners are returned from the call, we make a lot of requests to github API, I will need to implement a local cache to speed up the lambda execution time.

@npalm
Copy link
Member

npalm commented Sep 15, 2020

Been testing this PR on our own org, I've discovered that the lambda ends up timing out because if a large amount of runners are returned from the call, we make a lot of requests to github API, I will need to implement a local cache to speed up the lambda execution time.

How many runners do you have on averagen?

@HenryNguyen5
Copy link
Contributor Author

~30 or so

@HenryNguyen5
Copy link
Contributor Author

@npalm added function memoization, should reduce execution time by around 75% per my testing

@HenryNguyen5
Copy link
Contributor Author

@npalm @gertjanmaas anything i can do here to get this merged in?

Copy link
Contributor

@gertjanmaas gertjanmaas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I want to do a test before merging this back. Hope I have some time for this soon.

Copy link
Member

@npalm npalm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, reviewd and tested. All good. thanks once again 👍

@npalm npalm merged commit 4f18ac5 into github-aws-runners:develop Oct 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

If there are more than 100 runners registered, scale-down fails
3 participants