Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added pyenchant/pylint automated spell checking #159

Merged
merged 5 commits into from
Apr 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ pyinstaller = "==4.2"
pywin32-ctypes = "==0.2.0" # Eli (3/1/21): this must be explicitly specified when using a Linux dev environment so that it will be included in a Windows build environment
pefile = "==2019.4.18" # Eli (3/1/21): this must be explicitly specified when using a Linux dev environment so that it will be included in a Windows build environment
atomicwrites = "==1.4.0" # Eli (3/1/21): this must be explicitly specified when using a Linux dev environment so that it will be included in a Windows build environment
pyenchant = "==3.2.0"

[packages]
flask = "==1.1.2"
Expand Down
24 changes: 17 additions & 7 deletions Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

73 changes: 73 additions & 0 deletions src/.pylint-spelling-private-dict
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
pragma
docstring
mantarray
backend
frontend
hacky
args
noqa
unhandled
dicts
xem
bytearray
bytearrays
barcode
mypy
kwarg
centimilliseconds
spi
uploader
zimports
bool
barcodes
timepoint
dataset
mcu
cython
microcontroller
subprocess
subprocesses
cryptographic
sawtooth
init
nosec
numpy
json
pyinstaller
Werkzeug
werkzeug
untyped
getter
unflatten
subfunction
csv
exe
mixin
hardcoded
eval
pyserial
dma
timesteps
cms
semver
CLI
pytest
Github
github
centimillisecond
Entrypoint
entrypoint
intra
EP
STM
teardown
freezegun
fmt
linux
Butterworth
endian
ADC
parametrization
parametrizing
parametrize
parametrized
2 changes: 1 addition & 1 deletion src/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import mantarray_desktop_app

if __name__ == "__main__":
multiprocessing.freeze_support() # This line is not unit tested (Eli 12/24/19 unsure how to test something in this codeblock), but is critical to allow multiprocessing to work in complied PyInstaller bundles https://kite.com/python/docs/multiprocessing.freeze_support
multiprocessing.freeze_support() # This line is not unit tested (Eli 12/24/19 unsure how to test something in this code block), but is critical to allow multiprocessing to work in complied PyInstaller bundles https://kite.com/python/docs/multiprocessing.freeze_support
multiprocessing.set_start_method(
"spawn"
) # This line is not explicitly unit tested, but an error will be thrown in the main.main script if it is not executed. Windows only allows this start method, so need to make sure that in all development environments it is being tested in this mode.
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@
SUBPROCESS_SHUTDOWN_TIMEOUT_SECONDS = 1
SUBPROCESS_POLL_DELAY_SECONDS = 0.025

SECONDS_TO_WAIT_WHEN_POLLING_QUEUES = 0.02 # Due to the unreliablity of the .empty() .qsize() methods in queues, switched to a .get(timeout=) approach for polling the queues in the subprocesses. Eli (10/26/20): 0.01 seconds was still causing sporadic failures in Linux CI in Github, so bumped to 0.02 seconds.
SECONDS_TO_WAIT_WHEN_POLLING_QUEUES = 0.02 # Due to the unreliability of the :method:`.empty()` :method:`.qsize()` methods in queues, switched to a :method:`.get(timeout=)` approach for polling the queues in the subprocesses. Eli (10/26/20): 0.01 seconds was still causing sporadic failures in Linux CI in Github, so bumped to 0.02 seconds.

# Serial Communication Values
SERIAL_COMM_BAUD_RATE = 4e6
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/data_analyzer.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ class DataAnalyzerProcess(InfiniteProcess):
"""Process that analyzes data.

Args:
board_queues: A tuple (the max number of board connections should be pre-defined, so not a mutable list) of tuples of 2 queues. The first queue is for incoming data for that board that should be analyzed. The second queue is for finalized outgoing data to main process
board_queues: A tuple (the max number of board connections should be predefined, so not a mutable list) of tuples of 2 queues. The first queue is for incoming data for that board that should be analyzed. The second queue is for finalized outgoing data to main process
from_main_queue: a queue of communication from the main process
to_main_queue: a queue to put general communication back to main
fatal_error_reporter: a queue to report fatal errors back to the main process
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/fifo_read_producer.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def produce_data(num_cycles: int, starting_sample_index: int) -> bytearray:
)
sample_indices.append(sample_index)
sawtooth_indices.append(sample_index / FIFO_READ_PRODUCER_SAWTOOTH_PERIOD)
# generate sawtooth vals
# generate sawtooth values
sawtooth_vals = signal.sawtooth(sawtooth_indices, width=0.5)
# generate bytearray data
data = bytearray(0)
Expand Down
4 changes: 3 additions & 1 deletion src/mantarray_desktop_app/fifo_simulator.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ class RunningFIFOSimulator(FrontPanelSimulator, MantarrayFrontPanelMixIn):
"""Simulate a running Mantarray machine with OK board.

Args:
simulated_response_queues: dictionary where the ultimate leaves should be multiprocessing_utils.SimpleMultiprocessingQueue objects. These values are popped off the end of the queue and returned as if coming from the XEM. The 'wire_outs' key should contain a sub-dict with keys of integer values representing the ep addresses.
simulated_response_queues: dictionary where the ultimate leaves should be multiprocessing_utils.SimpleMultiprocessingQueue objects.
These values are popped off the end of the queue and returned as if coming from the XEM.
The 'wire_outs' key should contain a sub-dict with keys of integer values representing the EP addresses.
"""

default_device_id = "M02001900Mantarray Simulator"
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/file_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ class FileWriterProcess(InfiniteProcess):
"""Process that writes data to disk.

Args:
board_queues: A tuple (the max number of board connections should be pre-defined, so not a mutable list) of tuples of 2 queues. The first queue is for incoming data for that board that should be saved to disk. The second queue is for outgoing data for that board that has been saved to disk.
board_queues: A tuple (the max number of board connections should be predefined, so not a mutable list) of tuples of 2 queues. The first queue is for incoming data for that board that should be saved to disk. The second queue is for outgoing data for that board that has been saved to disk.
from_main_queue: a queue of communication from the main process
to_main_queue: a queue to put general communication back to main (including file names of finished files into so the uploader can begin uploading)
fatal_error_reporter: a queue to report fatal errors back to the main process
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/instrument_comm.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class InstrumentCommProcess(InfiniteProcess, metaclass=abc.ABCMeta):
"""Process that controls communication with Mantarray instruments.

Args:
board_queues: A tuple (the max number of instrument board connections should be pre-defined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely fo raw data) to the process that controls writing to disk.
board_queues: A tuple (the max number of instrument board connections should be predefined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely of raw data) to the process that controls writing to disk.
fatal_error_reporter: A queue that reports back any unhandled errors that have caused the process to stop.
suppress_setup_communication_to_main: if set to true (often during unit testing), messages during the _setup_before_loop will not be put into the queue to communicate back to the main process
"""
Expand Down
6 changes: 3 additions & 3 deletions src/mantarray_desktop_app/mc_comm.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ class McCommunicationProcess(InstrumentCommProcess):
"""Process that controls communication with the Mantarray Beta 2 Board(s).

Args:
board_queues: A tuple (the max number of MC board connections should be pre-defined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely fo raw data) to the process that controls writing to disk.
board_queues: A tuple (the max number of MC board connections should be predefined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely of raw data) to the process that controls writing to disk.
fatal_error_reporter: A queue that reports back any unhandled errors that have caused the process to stop.
suppress_setup_communication_to_main: if set to true (often during unit tests), messages during the _setup_before_loop will not be put into the queue to communicate back to the main process
"""
Expand Down Expand Up @@ -268,7 +268,7 @@ def _commands_for_each_run_iteration(self) -> None:
def _process_next_communication_from_main(self) -> None:
"""Process the next communication sent from the main process.

Will just return if no comms from main in queue.
Will just return if no communications from main in queue.
"""
input_queue = self._board_queues[0][0]
try:
Expand Down Expand Up @@ -480,7 +480,7 @@ def _register_magic_word(self, board_idx: int) -> None:
break
sleep(1)
else:
# if the entire period has passed and no more bytes are available an error has occured with the Mantarray that is considered fatal
# if the entire period has passed and no more bytes are available an error has occurred with the Mantarray that is considered fatal
raise SerialCommPacketRegistrationTimoutError(magic_word_test_bytes)
# read more bytes until the magic word is registered, the timeout value is reached, or the maximum number of bytes are read
num_bytes_checked = 0
Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/mc_simulator.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ def _send_data_packet(
timestamp, module_id, packet_type, data_to_send
)
if truncate:
trunc_index = random.randint( # nosec B311 # Tanner (2/4/21): Bandit blacklisted this psuedo-random generator for cryptographic security reasons that do not apply to the desktop app.
trunc_index = random.randint( # nosec B311 # Tanner (2/4/21): Bandit blacklisted this pseudo-random generator for cryptographic security reasons that do not apply to the desktop app.
0, len(data_packet) - 1
)
data_packet = data_packet[trunc_index:]
Expand Down
4 changes: 2 additions & 2 deletions src/mantarray_desktop_app/ok_comm.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ def _create_activate_trigger_in_callable(
def _comm_delay(
communication: Dict[str, Any],
) -> str:
"""Pause comms to XEM for given number of milliseconds."""
"""Pause communications to XEM for given number of milliseconds."""
num_milliseconds = communication["num_milliseconds"]
sleep_val = num_milliseconds / 1000
time.sleep(sleep_val)
Expand Down Expand Up @@ -462,7 +462,7 @@ class OkCommunicationProcess(InstrumentCommProcess):
"""Process that controls communication with the OpalKelly Board(s).

Args:
board_queues: A tuple (the max number of board connections should be pre-defined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely fo raw data) to the process that controls writing to disk.
board_queues: A tuple (the max number of board connections should be predefined, so not a mutable list) of tuples of 3 queues. The first queue is for input/communication from the main thread to this sub process, second queue is for communication from this process back to the main thread. Third queue is for streaming communication (largely of raw data) to the process that controls writing to disk.
fatal_error_reporter: A queue that reports back any unhandled errors that have caused the process to stop.
suppress_setup_communication_to_main: if set to true (often during unit testing), messages during the _setup_before_loop will not be put into the queue to communicate back to the main process
"""
Expand Down
4 changes: 2 additions & 2 deletions src/mantarray_desktop_app/process_monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ def _check_and_handle_server_to_main_queue(self) -> None:

msg: str
if "mantarray_nickname" in communication:
# Tanner (1/20/21): items in communication dict are used after this log message is generated, so need to creat a copy of the dict when redacting info
# Tanner (1/20/21): items in communication dict are used after this log message is generated, so need to create a copy of the dict when redacting info
comm_copy = copy.deepcopy(communication)
comm_copy["mantarray_nickname"] = "*" * len(comm_copy["mantarray_nickname"])
msg = f"Communication from the Server: {comm_copy}"
Expand Down Expand Up @@ -288,7 +288,7 @@ def _check_and_handle_instrument_comm_to_main_queue(self) -> None:

msg: str
if "mantarray_nickname" in communication:
# Tanner (1/20/21): items in communication dict are used after this log message is generated, so need to creat a copy of the dict when redacting info
# Tanner (1/20/21): items in communication dict are used after this log message is generated, so need to create a copy of the dict when redacting info
comm_copy = copy.deepcopy(communication)
comm_copy["mantarray_nickname"] = "*" * len(comm_copy["mantarray_nickname"])
msg = f"Communication from the OpalKelly Controller: {comm_copy}"
Expand Down
4 changes: 2 additions & 2 deletions src/mantarray_desktop_app/request_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
class MantarrayRequestHandler(WSGIRequestHandler):
"""Almost identical to WSGIRequestHandler.

This handler will remove sensitive params from log messages.
This handler will remove sensitive parameters from log messages.
"""

# pylint: disable=arguments-differ # Tanner (1/20/21): The original param was `type` instead of `type_` which overrides the python builtin `type`
Expand All @@ -31,7 +31,7 @@ def log(self, type_: str, message: str, *args: Any) -> None:
args_list = list(args)
args_list[0] = scrubbed_msg
args = tuple(args_list)
# Eli (3/9/21): Since Flask is running in multi-threaded mode, it might be possible that some log messages get garbled. It's not immediatley clear if Flask itself prevents this, or if the liklihood is prohibitively low to not worry about it, ...or what a robust way to pass the same threading.Lock() to this method as exists in the ServerThread itself. So for now we're not worrying about locking here and we'll see if it causes any issues with garbled logging.
# Eli (3/9/21): Since Flask is running in multi-threaded mode, it might be possible that some log messages get garbled. It's not immediately clear if Flask itself prevents this, or if the likelihood is prohibitively low to not worry about it, ...or what a robust way to pass the same threading.Lock() to this method as exists in the ServerThread itself. So for now we're not worrying about locking here and we'll see if it causes any issues with garbled logging.
_log(
type_,
f"{self.address_string()} - - {message}\n", # type: ignore # Tanner (1/21/20): mypy is complaining that `address_string` is untyped
Expand Down
8 changes: 4 additions & 4 deletions src/mantarray_desktop_app/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ def get_server_address_components() -> Tuple[str, str, int]:
"""Get Flask server address components.

Returns:
protocol (i.e. http), host (i.e. 127.0.0.1), port (i.e. 4567)
protocol (i.e. HTTP), host (i.e. 127.0.0.1), port (i.e. 4567)
"""
try:
port_number = get_the_server_thread().get_port_number()
Expand Down Expand Up @@ -668,7 +668,7 @@ def queue_activate_trigger_in() -> Response:

@flask_app.route("/insert_xem_command_into_queue/comm_delay", methods=["GET"])
def queue_comm_delay() -> Response:
"""Queue up a command delay comms to the XEM for a given period of time.
"""Queue a command delay communication to the XEM for a period of time.

Mainly to be used in XEM scripting when delays between commands are necessary.

Expand Down Expand Up @@ -728,7 +728,7 @@ def queue_set_device_id() -> Response:

Do not use this route to set Mantarray Device Nicknames or Serial Numbers.

This route should be used cautiously as it will overwrite an exisiting Mantarray serial number / ID stored in the XEM.
This route should be used cautiously as it will overwrite an existing Mantarray serial number / ID stored in the XEM.

Can be invoked by: curl http://localhost:4567/insert_xem_command_into_queue/set_device_id?new_id=""
"""
Expand Down Expand Up @@ -1007,7 +1007,7 @@ def after_request(response: Response) -> Response:
return response


# TODO (Eli 11/3/20): refactor stdlib utils to separate some of the more generic multiprocessing functionality out of the "InfiniteLooping" mixin so that it could be included here without all the other things
# TODO (Eli 11/3/20): refactor :package:`stdlib-utils` to separate some of the more generic multiprocessing functionality out of the "InfiniteLooping" mixin so that it could be included here without all the other things
class ServerThread(InfiniteThread):
"""Thread to run the Flask server."""

Expand Down
2 changes: 1 addition & 1 deletion src/mantarray_desktop_app/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ def get_current_software_version() -> str:


def _trim_barcode(barcode: str) -> str:
"""Trim the trailing 1 or 2 ASCII NUL (0x00) chars off barcode."""
"""Trim the trailing 1 or 2 ASCII NULL (0x00) characters off barcode."""
if barcode[11] != chr(0):
return barcode
if barcode[10] != chr(0):
Expand Down