Commit Graph
20 Commits
Author SHA1 Message Date
J. Nick KostonandGitHub 03eea7bd3f Avoid subprocess memory copy when c library supports posix_spawn (#87958)
* use posix spawn on alpine

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy all the memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided and subprocess creation does not
get discernibly slow the larger the Home Assistant
python process grows.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* missed some

* adjust more tests

* coverage
2023-02-13 09:02:51 -05:00
e4a78420b8 Enable Ruff PT013 (#86757)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2023-01-27 09:09:46 +01:00
atlflyerandGitHub da11cef29a Report error code in log when command fails (#74319) 2022-07-02 09:58:08 -04:00
G JohanssonandGitHub 3771c154fa Improve code quality command_line (#65333) 2022-02-12 08:19:37 -06:00
Gabriel RauterandGitHub d26275011a Add unique_id configuration variable to command_line integration (#58596) 2022-01-03 11:44:47 +01:00
Paulus SchoutsenandGitHub 31153ac155 Move fixtures part 1 (#58902) 2021-11-01 20:47:05 -07:00
jan iversenandGitHub 6a4f414236 Change HomeAssistantType to HomeAssistant (#49522) 2021-04-21 15:53:06 -10:00
Marc MuellerandGitHub 106dc4d28a Don't import stdlib typing types from helpers.typing (#49104) 2021-04-12 18:43:14 +02:00
Dermot DuffyandGitHub be8584c0bc Overhaul command_line tests (#46682) 2021-03-01 17:27:04 +01:00
Franck NijhofandGitHub 65cf2fcb6f Drop asynctest (#44746) 2021-01-01 22:31:56 +01:00
Paulus SchoutsenandGitHub 6de02fc1b9 Fix some more usages of asynctest (#39570) 2020-09-02 14:53:07 +02:00
Franck NijhofandGitHub 1c2ebdf307 Upgrade black to 20.8b1 (#39287) 2020-08-27 13:56:20 +02:00
J. Nick KostonandGitHub 215e3f2dab Add the ability to reload command_line platforms from yaml (#39262) 2020-08-25 19:52:36 -05:00
J. Nick KostonandGitHub 7b728b17f7 Add missing timeout to command_line platforms: cover, notify, switch (#38497)
* Add missing timeout to command_line platforms: cover, notify, switch

* add timeout test for notify
2020-08-04 17:00:02 -10:00
276f3afb00 Do async_setup_platform in background (#36244)
Co-authored-by: J. Nick Koston <nick@koston.org>
2020-05-31 22:18:30 -07:00
Franck NijhofandGitHub 906385172a String formatting improvements for tests (#33663) 2020-04-05 00:26:08 +02:00
Ville SkyttäandGitHub 5e2ba2eb77 Enable some more bandit checks (#30857)
* Enable B108 (hardcoded tmp dir), address findings

* Enable B602 (subprocess popen with shell), address findings

* Enable B604 (start process with shell), address findings

* Enable B306 (mktemp), B307 (eval), and B325 (tempnam), no issues to address
2020-01-20 18:44:55 +02:00
Bas NijholtandFabian Affolter 73c373a0f2 use isort to sort imports according to PEP8 for command_line (#29627) 2019-12-08 22:25:57 +05:30
Paulus Schoutsen 4de97abc3a Black 2019-07-31 12:25:30 -07:00
Penny WoodandPaulus Schoutsen f195ecca4b Consolidate all platforms that have tests (#22109)
* Moved climate components with tests into platform dirs.

* Updated tests from climate component.

* Moved binary_sensor components with tests into platform dirs.

* Updated tests from binary_sensor component.

* Moved calendar components with tests into platform dirs.

* Updated tests from calendar component.

* Moved camera components with tests into platform dirs.

* Updated tests from camera component.

* Moved cover components with tests into platform dirs.

* Updated tests from cover component.

* Moved device_tracker components with tests into platform dirs.

* Updated tests from device_tracker component.

* Moved fan components with tests into platform dirs.

* Updated tests from fan component.

* Moved geo_location components with tests into platform dirs.

* Updated tests from geo_location component.

* Moved image_processing components with tests into platform dirs.

* Updated tests from image_processing component.

* Moved light components with tests into platform dirs.

* Updated tests from light component.

* Moved lock components with tests into platform dirs.

* Moved media_player components with tests into platform dirs.

* Updated tests from media_player component.

* Moved scene components with tests into platform dirs.

* Moved sensor components with tests into platform dirs.

* Updated tests from sensor component.

* Moved switch components with tests into platform dirs.

* Updated tests from sensor component.

* Moved vacuum components with tests into platform dirs.

* Updated tests from vacuum component.

* Moved weather components with tests into platform dirs.

* Fixed __init__.py files

* Fixes for stuff moved as part of this branch.

* Fix stuff needed to merge with balloob's branch.

* Formatting issues.

* Missing __init__.py files.

* Fix-ups

* Fixup

* Regenerated requirements.

* Linting errors fixed.

* Fixed more broken tests.

* Missing init files.

* Fix broken tests.

* More broken tests

* There seems to be a thread race condition.
I suspect the logger stuff is running in another thread, which means waiting until the aio loop is done is missing the log messages.
Used sleep instead because that allows the logger thread to run. I think the api_streams sensor might not be thread safe.

* Disabled tests, will remove sensor in #22147

* Updated coverage and codeowners.
2019-03-18 23:07:39 -07:00