loschmidt.tilted_square_lattice gen+run #259

mpharrigan · 2022-01-14T18:59:20Z

Following #236 , add scripts for generating a QuantumExecutableGroup and a script for running on a simulated EngineProcessor.

This PR includes a very simple placeholder bash script for doing an "end-to-end" test that will be replaced with a more comprehensive solution when we have all the end-to-end parts committed.

This PR does not include the data analysis steps, which will follow.

This requires quantumlib/Cirq#4842

Change-Id: Ieb7aa8fd71bc3cfa1e5282eeca9ca30e2d7ff4cc

Change-Id: I309b28ec9804ce547ed8d73a44418c7c282d4ac5

Change-Id: I4101cdd4acfb8b0b707d9e65439598396929a641

Change-Id: Ibb8a404c61b1dc5527eae1b9f11dda17ff836fb2

Change-Id: Ia04c61885725bd0124f5530a1c1d3bf360e58a98

Change-Id: I0cc080a84512ceebe262c2d611013be3dc8b3b15

dstrain115 · 2022-01-28T13:37:53Z

recirq/algorithmic_benchmark_library.py

@@ -83,6 +83,12 @@ class BenchmarkConfig:
        spec_class=TiltedSquareLatticeLoschmidtSpec,
        executable_generator_func=get_all_tilted_square_lattice_executables,
        configs=[
+            BenchmarkConfig(
+                short_name='small-v1',


Is there a good place to put a synopsis about what each benchmark is/does? Either as a line comment or as a structured field? Once we have a bunch of these, it's going to be more difficult to keep track of what each of them does.

good idea, added

dstrain115 · 2022-01-28T13:39:57Z

recirq/cirqflow/run_utils.py

+    i = 1
+    while True:
+        run_id = fmt.format(i=i, date=datetime.date.today().isoformat())
+        if not os.path.exists(f'{base_data_dir}/{run_id}'):


Optional: Not sure it matters much, but do you think it would be faster to list the directory and just check the results for each run rather than make O(n) separate os calls? (Not sure how many runs we expect to do, but maybe could be a lot of them?)

Yes, this is good to keep in mind. I'm imagining users will be coming up with meaningful run_id or run_id templates (particularly using the {date} option) so that we won't spend much time searching.

I'll punt on this since I think it would be straightforward to transparently update if and when it becomes a problem.

dstrain115 · 2022-01-28T13:41:27Z

recirq/otoc/loschmidt/tilted_square_lattice/end-to-end-test.sh

@@ -0,0 +1,3 @@
+python gen-small-v1.py


Worth a comment to explain this file? Also, does this file need license info?

added.

Worth noting that I have much more robust plans for doing end-to-end (ie integration testing) once more of the benchmarks are in place; but I figured it wouldn't hurt to have this baby bash script for the time being.

dstrain115 · 2022-01-28T13:41:40Z

recirq/otoc/loschmidt/tilted_square_lattice/gen-small-v1.py

@@ -0,0 +1,22 @@
+import numpy as np


Add license info.

dstrain115 · 2022-01-28T13:41:54Z

recirq/otoc/loschmidt/tilted_square_lattice/run-simulator.py

@@ -0,0 +1,24 @@
+import cirq


Add license info.

dstrain115 · 2022-01-28T13:42:31Z

recirq/otoc/loschmidt/tilted_square_lattice/gen-small-v1.py

+FN = 'loschmidt.tilted_square_lattice.small-v1.json.gz'
+
+
+def main():


Should add module level docstring to explain what this is and what the valid args and/or example command are.

added (using verbiage from the new description field in the algorithmic_benchmarking_library card catalog)

dstrain115 · 2022-01-28T13:43:24Z

recirq/otoc/loschmidt/tilted_square_lattice/gen-small-v1.py

+from recirq.otoc.loschmidt.tilted_square_lattice import \
+    get_all_tilted_square_lattice_executables
+
+FN = 'loschmidt.tilted_square_lattice.small-v1.json.gz'


What is this? Avoid abbreviations, especially when ambiguous (filename vs function).
Suggest OUTPUT_FILENAME or similar?

changed.

also I think at some point this filename should be fetched maybe from the algorithmic_benchmarking_library "card catalog" module but I haven't completely thought through how that would work.

dstrain115 · 2022-01-28T13:44:04Z

recirq/otoc/loschmidt/tilted_square_lattice/gen-small-v1.py

+        macrocycle_depths=np.arange(0, 4 + 1, 1))
+    print(len(exes), 'executables')
+
+    cirq.to_json_gzip(exes, FN)


Should the output filename be a flag?

The current design is to have a 1:1 correspondence between these generation scripts and the serialized QuantumExecutableGroup. I consider this file to be a configuration file like a yaml or textproto file. I could just use a yaml file or something; but I find often times the schema of configuration files gets extensions bolted on to it to handle new ways of configuring things so here I'm just using a real scripting language from the start.

Remember: we need to carefully control the benchmarks we run for accurate comparison over time. If you want to run an ad-hoc benchmark with your own parameter choices, you should likely create a new script to do what you want.

dstrain115 · 2022-01-28T13:44:37Z

recirq/otoc/loschmidt/tilted_square_lattice/run-simulator.py

+
+assert TiltedSquareLatticeLoschmidtSpec, 'register deserializer'
+
+FN = 'loschmidt.tilted_square_lattice.small-v1.json.gz'


Same comments about FN and docstrings in this file as well.

Change-Id: I6751482d16609c1fa1aa6ba9f5f039d8351926d8

Change-Id: I29c347d6d84a616f4f2937f70ef742d35c5457e9

Change-Id: I25c45fbfdb4889d2a3a991e96cbfc8225ab6a206

mpharrigan · 2022-01-28T19:33:34Z

thanks for the review @dstrain115, PTAL

dstrain115 · 2022-02-02T23:18:34Z

recirq/algorithmic_benchmark_library.py

@@ -48,6 +48,7 @@ class AlgorithmicBenchmark:
    spec_class: Type['ExecutableSpec']
    executable_generator_func: Callable[[Any], 'QuantumExecutableGroup']
    configs: List['BenchmarkConfig']
+    description: Optional[str] = None

    def as_strings(self):


Not related, but noticed it was missing return type.

Change-Id: I343d00ccbef48fab2667746858c8a256772bb942

Change-Id: Ie1e68b97379eeb9080e05aa9c56ba96b1b42eee3

Change-Id: Iafdd325501ffe60a53828b369564a8512d245916

mpharrigan added otoc ftb/bl labels Jan 14, 2022

mpharrigan requested a review from dstrain115 January 14, 2022 18:59

mpharrigan added 4 commits January 27, 2022 13:58

[cirqflow] run_utils - get_unique_run_id

30bfafd

Change-Id: Ieb7aa8fd71bc3cfa1e5282eeca9ca30e2d7ff4cc

loschmidt.tilted_square_lattice problem generation and simulated running

5ecd886

Change-Id: I309b28ec9804ce547ed8d73a44418c7c282d4ac5

update card catalog

61a8bff

Change-Id: I4101cdd4acfb8b0b707d9e65439598396929a641

Fixup use cirq_google stuff

fd92a06

Change-Id: Ibb8a404c61b1dc5527eae1b9f11dda17ff836fb2

mpharrigan force-pushed the 2022-01-loschmidt-run branch from 43bda6d to fd92a06 Compare January 27, 2022 22:01

mpharrigan added 2 commits January 27, 2022 14:01

missing init

01c1b1e

Change-Id: Ia04c61885725bd0124f5530a1c1d3bf360e58a98

Test run_utils

d4a94fb

Change-Id: I0cc080a84512ceebe262c2d611013be3dc8b3b15

dstrain115 reviewed Jan 28, 2022

View reviewed changes

mpharrigan added 3 commits January 28, 2022 11:16

add descriptions to library library

990d354

Change-Id: I6751482d16609c1fa1aa6ba9f5f039d8351926d8

license and description of end-to-end-test.sh

975603e

Change-Id: I29c347d6d84a616f4f2937f70ef742d35c5457e9

improve scripts

9fd1023

Change-Id: I25c45fbfdb4889d2a3a991e96cbfc8225ab6a206

mpharrigan mentioned this pull request Jan 30, 2022

loschmidt.tilted_square_lattice analysis utilities and notebook walkthrough #261

Merged

dstrain115 approved these changes Feb 2, 2022

View reviewed changes

mpharrigan added 2 commits February 16, 2022 11:17

algo benchmark ret type

3eddc0a

Change-Id: I343d00ccbef48fab2667746858c8a256772bb942

Merge remote-tracking branch 'github/master' into 2022-01-loschmidt-run

69f2177

Change-Id: Ie1e68b97379eeb9080e05aa9c56ba96b1b42eee3

mpharrigan mentioned this pull request Feb 16, 2022

Pytest skip breakage #266

Merged

Merge remote-tracking branch 'github/master' into 2022-01-loschmidt-run

76ec267

Change-Id: Iafdd325501ffe60a53828b369564a8512d245916

mpharrigan merged commit 783452f into quantumlib:master Feb 16, 2022

		FN = 'loschmidt.tilted_square_lattice.small-v1.json.gz'


		def main():


		assert TiltedSquareLatticeLoschmidtSpec, 'register deserializer'

		FN = 'loschmidt.tilted_square_lattice.small-v1.json.gz'

loschmidt.tilted_square_lattice gen+run #259

loschmidt.tilted_square_lattice gen+run #259

Uh oh!

Conversation

mpharrigan commented Jan 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mpharrigan commented Jan 28, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mpharrigan commented Jan 14, 2022 •

edited

Loading