Skip to content

ENH: linkcheck: also write all links to output.json #7103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 1, 2020

Conversation

westurner
Copy link
Contributor

@westurner westurner commented Feb 7, 2020

Subject: Write linkcheck output to JSON lines in _build/linkcheck/output.json

This is off 1.8.5. If it would be considered, I'd be glad to rebase it onto master.

(edit)

This is now off master.

Feature or Bugfix

  • Feature

Purpose

  • Write linkcheck output to JSON lines in _build/linkcheck/output.json
    (in order to eventually create a tool to prompt to fix links)

Detail

Relates

Copy link
Member

@tk0miya tk0miya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can accept your idea. But it seems conflicted with latest code. Could you rebase this into HEAD?

@tk0miya tk0miya added builder type:proposal a feature suggestion labels Feb 8, 2020
@tk0miya tk0miya added this to the 3.0.0 milestone Feb 8, 2020
@westurner westurner force-pushed the linkcheck_json_output branch from 5c818cb to 7a954ac Compare February 8, 2020 22:46
@westurner westurner changed the title ENH: linkcheck.py: also write all links to output.json ENH: linkcheck: also write all links to output.json Feb 8, 2020
@westurner
Copy link
Contributor Author

Seems to build fine here with tox -e py37 -- -v --pdb ./tests/test_build_linkcheck.py.
Do I need to do something to get the CI to run again?

@westurner
Copy link
Contributor Author

westurner commented Feb 8, 2020

The output order is nondeterministic presumably due to variance in network latency.
I could just drop the row numbers in tests/test_build_linkcheck.py and revert to testing for string match?

@westurner westurner requested a review from tk0miya February 9, 2020 00:02
@westurner
Copy link
Contributor Author

... The python: nightly job is returning this, which is probably not an error with this PR?

$ if [ $IS_PYTHON = true ]; then pip install -U tox codecov; fi
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.9-dev/bin/pip", line 6, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'

https://travis-ci.org/sphinx-doc/sphinx/jobs/648641307

Copy link
Member

@tk0miya tk0miya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with nits.

@tk0miya
Copy link
Member

tk0miya commented Feb 12, 2020

In addition, please merge master branch to this branch to fix the CI error.

@westurner westurner force-pushed the linkcheck_json_output branch 4 times, most recently from e14970b to 88e312c Compare February 12, 2020 20:14
@westurner
Copy link
Contributor Author

I'm tempted to start an http.server as a @pytest.fixture in order to eliminate the multiple unnecessary HTTP requests to google on every test run due to the links to google in links.txt. If that's okay, would either of these or another solution be preferable?

* TST: linkcheck: make tests more flexible
* CLN: linkcheck: flake8, mypy
* REF: linkcheck: docpath->filename, write_jsonline->write_linkstat
* REF: linkcheck: remove redundant call to doc2path
* TST: linkcheck: show JSON obj structure in test
* REF: linkcheck: remove docname from JSON obj because it's redundant (use path2doc(filename) if necessary)
* TST: linkcheck: don't test row[info] output (see comments for examples)
@westurner westurner force-pushed the linkcheck_json_output branch from 88e312c to fd94270 Compare February 12, 2020 21:30
@westurner
Copy link
Contributor Author

westurner commented Feb 12, 2020

This is otherwise rebased onto latest master with:

git checkout master
git pull upstream master
git checkout linkcheck_json_output
git rebase -i master

And squashed

@westurner westurner requested a review from tk0miya February 12, 2020 22:08
@tk0miya tk0miya changed the base branch from master to 3.x March 1, 2020 08:42
@tk0miya tk0miya merged commit 64d51a1 into sphinx-doc:3.x Mar 1, 2020
@tk0miya
Copy link
Member

tk0miya commented Mar 1, 2020

Merged. Thank you for your work!

tk0miya added a commit that referenced this pull request Mar 1, 2020
@westurner
Copy link
Contributor Author

westurner commented Mar 1, 2020 via email

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
builder type:proposal a feature suggestion
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants