2022-06-02 09:58:46 +02:00
|
|
|
# This file contains type hints that can be prepended to Nix test scripts so they can be type
|
|
|
|
# checked.
|
|
|
|
|
|
|
|
from test_driver.driver import Driver
|
|
|
|
from test_driver.vlan import VLan
|
|
|
|
from test_driver.machine import Machine
|
2024-04-08 16:17:06 +02:00
|
|
|
from test_driver.logger import AbstractLogger
|
2022-06-02 09:58:46 +02:00
|
|
|
from typing import Callable, Iterator, ContextManager, Optional, List, Dict, Any, Union
|
|
|
|
from typing_extensions import Protocol
|
|
|
|
from pathlib import Path
|
nixos/test-driver: integrate Python `unittest` assertions
Replaces / Closes #345948
I tried to integrate `pytest` assertions because I like the reporting,
but I only managed to get the very basic thing and even that was messing
around a lot with its internals.
The approach in #345948 shifts too much maintenance effort to us, so
it's not really desirable either.
After discussing with Benoit on Ocean Sprint about this, we decided that
it's probably the best compromise to integrate `unittest`: it also
provides good diffs when needed, but the downside is that existing tests
don't benefit from it.
This patch essentially does the following things:
* Add a new global `t` that is an instance of a `unittest.TestCase`
class. I decided to just go for `t` given that e.g.
`tester.assertEqual` (or any other longer name) seems quite verbose.
* Use a special class for errors that get special treatment:
* The traceback is minimized to only include frames from the
testScript: in this case I don't really care about anything else and
IMHO that's just visual noise.
This is not the case for other exceptions since these may indicate a
bug and then people should be able to send the full traceback to the
maintainers.
* Display the error, but with `!!!` as prefix to make sure it's
easier to spot in between other logs.
This looks e.g. like
!!! Traceback (most recent call last):
!!! File "<string>", line 7, in <module>
!!! foo()
!!! File "<string>", line 5, in foo
!!! t.assertEqual({"foo":[1,2,{"foo":"bar"}]},{"foo":[1,2,{"bar":"foo"}],"bar":[1,2,3,4,"foo"]})
!!!
!!! NixOSAssertionError: {'foo': [1, 2, {'foo': 'bar'}]} != {'foo': [1, 2, {'bar': 'foo'}], 'bar': [1, 2, 3, 4, 'foo']}
!!! - {'foo': [1, 2, {'foo': 'bar'}]}
!!! + {'bar': [1, 2, 3, 4, 'foo'], 'foo': [1, 2, {'bar': 'foo'}]}
cleanup
kill machine (pid 9)
qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/wz0j2zi02rvnjiz37nn28h3gfdq61svz-python3-3.12.9/bin/python3.12)
kill vlan (pid 7)
(finished: cleanup, in 0.00 seconds)
Co-authored-by: bew <bew@users.noreply.github.com>
2025-03-17 17:27:59 +00:00
|
|
|
from unittest import TestCase
|
2022-06-02 09:58:46 +02:00
|
|
|
|
|
|
|
|
|
|
|
class RetryProtocol(Protocol):
|
|
|
|
def __call__(self, fn: Callable, timeout: int = 900) -> None:
|
|
|
|
raise Exception("This is just type information for the Nix test driver")
|
|
|
|
|
|
|
|
|
|
|
|
class PollingConditionProtocol(Protocol):
|
|
|
|
def __call__(
|
|
|
|
self,
|
|
|
|
fun_: Optional[Callable] = None,
|
|
|
|
*,
|
|
|
|
seconds_interval: float = 2.0,
|
|
|
|
description: Optional[str] = None,
|
|
|
|
) -> Union[Callable[[Callable], ContextManager], ContextManager]:
|
|
|
|
raise Exception("This is just type information for the Nix test driver")
|
|
|
|
|
|
|
|
|
2024-02-26 13:30:43 +03:00
|
|
|
class CreateMachineProtocol(Protocol):
|
|
|
|
def __call__(
|
|
|
|
self,
|
2024-02-27 23:14:15 +03:00
|
|
|
start_command: str | dict,
|
2024-02-26 13:30:43 +03:00
|
|
|
*,
|
|
|
|
name: Optional[str] = None,
|
|
|
|
keep_vm_state: bool = False,
|
|
|
|
) -> Machine:
|
|
|
|
raise Exception("This is just type information for the Nix test driver")
|
|
|
|
|
|
|
|
|
2022-06-02 09:58:46 +02:00
|
|
|
start_all: Callable[[], None]
|
|
|
|
subtest: Callable[[str], ContextManager[None]]
|
|
|
|
retry: RetryProtocol
|
|
|
|
test_script: Callable[[], None]
|
|
|
|
machines: List[Machine]
|
|
|
|
vlans: List[VLan]
|
|
|
|
driver: Driver
|
2024-04-08 16:17:06 +02:00
|
|
|
log: AbstractLogger
|
2024-02-26 13:30:43 +03:00
|
|
|
create_machine: CreateMachineProtocol
|
2022-06-02 09:58:46 +02:00
|
|
|
run_tests: Callable[[], None]
|
|
|
|
join_all: Callable[[], None]
|
|
|
|
serial_stdout_off: Callable[[], None]
|
|
|
|
serial_stdout_on: Callable[[], None]
|
|
|
|
polling_condition: PollingConditionProtocol
|
nixos/test-driver: integrate Python `unittest` assertions
Replaces / Closes #345948
I tried to integrate `pytest` assertions because I like the reporting,
but I only managed to get the very basic thing and even that was messing
around a lot with its internals.
The approach in #345948 shifts too much maintenance effort to us, so
it's not really desirable either.
After discussing with Benoit on Ocean Sprint about this, we decided that
it's probably the best compromise to integrate `unittest`: it also
provides good diffs when needed, but the downside is that existing tests
don't benefit from it.
This patch essentially does the following things:
* Add a new global `t` that is an instance of a `unittest.TestCase`
class. I decided to just go for `t` given that e.g.
`tester.assertEqual` (or any other longer name) seems quite verbose.
* Use a special class for errors that get special treatment:
* The traceback is minimized to only include frames from the
testScript: in this case I don't really care about anything else and
IMHO that's just visual noise.
This is not the case for other exceptions since these may indicate a
bug and then people should be able to send the full traceback to the
maintainers.
* Display the error, but with `!!!` as prefix to make sure it's
easier to spot in between other logs.
This looks e.g. like
!!! Traceback (most recent call last):
!!! File "<string>", line 7, in <module>
!!! foo()
!!! File "<string>", line 5, in foo
!!! t.assertEqual({"foo":[1,2,{"foo":"bar"}]},{"foo":[1,2,{"bar":"foo"}],"bar":[1,2,3,4,"foo"]})
!!!
!!! NixOSAssertionError: {'foo': [1, 2, {'foo': 'bar'}]} != {'foo': [1, 2, {'bar': 'foo'}], 'bar': [1, 2, 3, 4, 'foo']}
!!! - {'foo': [1, 2, {'foo': 'bar'}]}
!!! + {'bar': [1, 2, 3, 4, 'foo'], 'foo': [1, 2, {'bar': 'foo'}]}
cleanup
kill machine (pid 9)
qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/wz0j2zi02rvnjiz37nn28h3gfdq61svz-python3-3.12.9/bin/python3.12)
kill vlan (pid 7)
(finished: cleanup, in 0.00 seconds)
Co-authored-by: bew <bew@users.noreply.github.com>
2025-03-17 17:27:59 +00:00
|
|
|
t: TestCase
|