Skip to content

618 check systems test rewrite #653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 15, 2024
Merged

618 check systems test rewrite #653

merged 3 commits into from
Jan 15, 2024

Conversation

dan-mm
Copy link
Contributor

@dan-mm dan-mm commented Jan 12, 2024

One thing I noticed while doing this is the following: here is the check_one_psu_provider function that is called in systems_check.py:

def check_one_psu_provider():
    metric_providers = utils.get_metric_providers(GlobalConfig().config).keys()
    energy_machine_providers = [provider for provider in metric_providers if ".energy" in provider and ".machine" in provider]
    return len(energy_machine_providers) <= 1

it checks for keys that have .energy and .machine in the key and determines those as psu providers. Why do we not check for keys that have .psu instead?

Copy link

github-actions bot commented Jan 12, 2024

Old Energy Estimation

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 20.6496 1021.84 3.37241 310
Measurement #1 20.9226 1021.84 3.37241 304

📈 Energy graph:

 
 7.77 ┤                                                        ╭╮
 7.17 ┤                                      ╭╮                ││
 6.57 ┤                                      ││                ││
 5.97 ┤                                     ╭╯│           ╭─╮  │╰╮╭╮
 5.37 ┤                                     │ │          ╭╯ ╰╮ │ ╰╯│
 4.77 ┤                                     │ ╰─╮╭─╮╭╮   │   │ │   │              ╭╮
 4.17 ┤     ╭─╮    ╭─╮  ╭╮╭╮ ╭──╮    ╭─╮    │   ╰╯ ╰╯│   │   ╰╮│   │        ╭╮╭╮  ││ ╭─╮ ╭╮╭╮                        ╭╮ ╭────╮                              ╭╮ ╭╮     ╭╮               ╭╮       ╭╮  ╭╮     ╭╮                              ╭╮        ╭╮                  ╭╮        ╭╮ ╭╮╭╮                          ╭
 3.57 ┤    ╭╯ ╰────╯ ╰──╯╰╯╰─╯  ╰────╯ ╰╮   │        ╰╮ ╭╯    ╰╯   │       ╭╯╰╯╰─╮││╭╯ ╰─╯╰╯╰────────────────────────╯╰─╯    ╰──────╮                    ╭──╯╰─╯╰──╮╭─╯╰───────────────╯╰───────╯╰──╯╰─────╯╰╮          ╭╮         ╭╮ ╭────╯╰───────╮│╰──────────────────╯╰────────╯╰─╯╰╯╰─╮        ╭─╮         ╭╮ ╭╯
 2.97 ┤    │                            │╭──╯         │ │          │       │     ╰╯╰╯                                               │                    │         ││                                        ╰╮        ╭╯│         │╰╮│             ╰╯                                     │        │ │         ││ │
 2.37 ┤    │                            ││            ╰╮│          │       │                                                        │                   ╭╯         ╰╯                                         │       ╭╯ ╰─╮       │ ││                                                    │       ╭╯ ╰╮        ││ │
 1.77 ┼────╯                            ╰╯             ╰╯          ╰───────╯                                                        ╰───────────────────╯                                                     ╰───────╯    ╰───────╯ ╰╯                                                    ╰───────╯   ╰────────╯╰─╯
                                                                                                                                                       Watts over time

@ArneTR
Copy link
Member

ArneTR commented Jan 12, 2024

One thing I noticed while doing this is the following: here is the check_one_psu_provider function that is called in systems_check.py:

def check_one_psu_provider():
    metric_providers = utils.get_metric_providers(GlobalConfig().config).keys()
    energy_machine_providers = [provider for provider in metric_providers if ".energy" in provider and ".machine" in provider]
    return len(energy_machine_providers) <= 1

it checks for keys that have .energy and .machine in the key and determines those as psu providers. Why do we not check for keys that have .psu instead?

The naming of the test is misleading. What we actually want to check is that we do not have two providers that provider energy and scope machine. Where the value is coming from is irrelevant. It can be a PSU, but also a PDU would be fine.

Please change the name of the test and maybe also the error message if that claims PSU is the problem rather than energy_ machine_ . Then fine to merge

@dan-mm
Copy link
Contributor Author

dan-mm commented Jan 15, 2024

done - will merge after I see tests pass

Copy link

Eco-CI Output:

Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run 19.7835 1296.85 3.31676 398
Measurement #1 20.032 1296.85 3.31676 392

📈 Energy graph:

 
 7.74 ┤                                                                                     ╭╮                             ╭╮
 7.14 ┤                                                                                     ││                             ││
 6.55 ┤                                                                                     │╰╮                       ╭╮   ││
 5.95 ┤                                                                                     │ │                       ││   ││╭╮
 5.35 ┤              ╭╮                                                       ╭╮            │ │                      ╭╯╰╮ ╭╯│││
 4.75 ┤              ││                                                       ││            │ │  ╭─╮╭╮              ╭╯  ╰╮│ ╰╯│
 4.16 ┤             ╭╯╰─╮  ╭───╮ ╭╮    ╭╮   ╭╮╭─╮    ╭─╮                      │╰╮          ╭╯ ╰──╯ ╰╯╰╮             │    ╰╯   │         ╭╮╭╮  ╭╮  ╭─╮ ╭╮╭╮    ╭╮                  ╭╮ ╭─────╮                              ╭╮ ╭╮     ╭╮         ╭╮    ╭╮  ╭─╮  ╭╮  ╭╮     ╭╮                                   ╭╮ ╭╮   ╭╮  ╭╮                                  ╭╮                                            ╭
 3.56 ┤      ╭╮ ╭╮ ╭╯   ╰──╯   ╰─╯╰──╮╭╯╰───╯╰╯ │    │ ╰──────────────────────╯ ╰──────────╯          │             │         ╰╮       ╭╯││╰──╯╰╮╭╯ ╰─╯╰╯╰────╯╰─────────────────╮│╰─╯     ╰────╮                      ╭──╯╰─╯╰──╮╭─╯╰─────────╯╰────╯╰──╯ ╰──╯╰──╯╰─────╯╰─╮         ╭─╮         ╭╮ ╭───╮╭───╯╰─╯╰───╯╰╮ │╰────╮╭─────╮╭─╮╭╮╭─╮╭───╮╭──────╮╭╯╰──╮╭────╮╭───────╮          ╭╮         ╭─╮╭─╯
 2.96 ┤      ││ │╰─╯                 ╰╯         ╰╮╭╮ │                                                │             │          │       │ ││     ╰╯                               ││             ╰╮                     │         ││                                         │         │ │         ││ │   ││             │ │     ╰╯     ╰╯ ││╰╯ ││   ╰╯      ││    ││    ││       ╰╮         ││         │ ││
 2.37 ┤      ││ │                                ╰╯│ │                                                │     ╭╮      │          │       │ ╰╯                                      ││              ╰─╮                  ╭╯         ╰╯                                         │         │ ╰─╮       │╰╮│   ╰╯             │ │               ╰╯   ╰╯           ╰╯    ╰╯    ││        │       ╭╮│╰─╮       │ ││
 1.77 ┼──────╯╰─╯                                  ╰─╯                                                ╰─────╯╰──────╯          ╰───────╯                                         ╰╯                ╰──────────────────╯                                                     ╰─────────╯   ╰───────╯ ╰╯                  ╰─╯                                             ╰╯        ╰───────╯╰╯  ╰───────╯ ╰╯
                                                                                                                                                                                                   Watts over time

@dan-mm dan-mm merged commit 1fd7f2f into main Jan 15, 2024
@dan-mm dan-mm deleted the 618-check-systems-test-rewrite branch January 15, 2024 09:51
@dan-mm dan-mm linked an issue Jan 15, 2024 that may be closed by this pull request
@dan-mm dan-mm mentioned this pull request Jan 18, 2024
ArneTR added a commit that referenced this pull request Jan 27, 2024
* main: (42 commits)
  update symlink for memory/energy/rapl source.c to reflect changed folder names
  renamed RAPL/MSR folders to lowercase, to match changes in config.yml and fix module import
  Added guard clause for client.py to not run Measurement Control when job is still running
  Bump uvicorn[standard] from 0.26.0 to 0.27.0 (#663)
  Bump scipy from 1.11.4 to 1.12.0 (#661)
  Bump pandas from 2.1.4 to 2.2.0 (#660)
  Bump psutil from 5.9.7 to 5.9.8 (#662)
  Normalized indents
  standardized test-config (#650)
  Bump orjson from 3.9.10 to 3.9.12 (#658)
  Bump docker/build-push-action from 2 to 5 (#552)
  Bump docker/login-action from 2 to 3 (#551)
  Bump uvicorn[standard] from 0.25.0 to 0.26.0 (#657)
  Variance test actually reflecting 10 inserts
  Clarifications about which STDDEV we use
  618 check systems test rewrite (#653)
  Updated starlette
  chore: add info about forking workflow for contributors (#656)
  Tests and better handling for detached process (#655)
  New sidebar (#654)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests are not always using test containers
2 participants