Commit Graph
57 Commits
Author SHA1 Message Date
Tatham OddieandGitHub 4fa9871080 Fix logbook entity_matches_only query mode (#55761)
The string matching template needs to match the same compact JSON format
as the data is now written in.
2021-09-06 13:39:39 -07:00
89e7983ee0 Add Blueprint foundation to Scripts (#48621)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-04-27 15:15:38 -07:00
Paulus SchoutsenandGitHub 04b335afe9 Allow filtering the logbook by context_id (#47783) 2021-03-12 09:04:02 -08:00
Michael ChisholmandGitHub e35e460e69 Use parent_id to find cause of logbook events with new contexts (#44416)
* Use parent_id to find cause of events with new contexts

When looking up the causing event for logbook display, use the `parent_id`
of the current context if the current context just points back to the
current event.

This now shows in the logbook the cause of an event in the case that a
component has created a new context from an existing context and tied
them together via the `Context.parent_id`.

* Fix exception when parent event not available

* Use async_Log_entry to avoid jump into executor
2021-01-07 20:27:03 -10:00
Franck NijhofandGitHub 65cf2fcb6f Drop asynctest (#44746) 2021-01-01 22:31:56 +01:00
taiyeogunsandGitHub 8f74a0c2e6 Convert logbook tests to pytest (#42289) 2020-10-26 09:18:02 +01:00
Philip AllgaierandGitHub dde6305549 Cleanup unused loggers (components N-Z + tests) (#41982) 2020-10-16 21:24:08 -05:00
J. Nick KostonandGitHub 49e5b665cf Convert final async_add_job to async_add_executor_job in logbook tests (#41552) 2020-10-09 15:52:06 +02:00
J. Nick KostonandGitHub 2076fc59c4 Remove message and domain from logbook state_changed events (#40070)
These are now handled by the frontend as we move closer
to making the logbook localized.
2020-09-30 11:29:51 -05:00
J. Nick KostonandGitHub d9ba32dc3f Setup recorder model relationships to avoid calling flush (#40467) 2020-09-30 13:11:43 +02:00
J. Nick KostonandGitHub aada6a1d88 Fix logbook with empty filter and remove unused code (#40565) 2020-09-30 13:10:11 +02:00
J. Nick KostonandGitHub a19e10c57a Add entity glob matching support to history (#40387) 2020-09-28 15:58:55 +02:00
J. Nick KostonandGitHub 7029345b9d Add support for selecting multiple entity ids from logbook (#40075) 2020-09-22 09:27:04 -05:00
J. Nick KostonandGitHub 15281f468e Extract the icon and state for logbook state changed events (#40039) 2020-09-14 09:35:19 -05:00
45c28dd9c5 Provide a logbook option entity_matches_only to optimize for single entity lookup (#39555)
* Provide a logbook option entity_matches_only to optimize for single entity id lookup

When entity_matches_only is provided, contexts and events that do not
contain the entity_id are not included in the logbook response.

* Update homeassistant/components/logbook/__init__.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* api only takes a single entity

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2020-09-02 10:32:08 -05:00
J. Nick KostonandGitHub dda4cf4d58 Tune logbook performance to accomodate recent changes (#39348) 2020-08-30 09:20:31 +02:00
J. Nick KostonandGitHub 6b7a7939d2 Include the first seen context data in the logbook api (#39194)
* Include the context_entity_id in the logbook api

context_entity_id is the first entity seen during
a time period that includes the context

* update test

* more of them

* include friendly name

* pylint wants a ternary

* Refactor

* performance

* fix homekit context

* Fix self describing events

* Fix external_events
2020-08-24 12:44:40 -05:00
J. Nick KostonandGitHub f5b628c04f Cleanup logbook tests to prevent failure on race condition (#37928) 2020-07-16 21:48:22 -07:00
J. Nick KostonandGitHub 9db6318122 Remove support for legacy logbook events created before 0.112 (#37822)
* Remove support for legacy logbook events created before 0.112

Reduce the complexity of the logbook code.  This
should also have a small performance boost.

* None is the default
2020-07-15 10:38:08 -10:00
J. Nick KostonandGitHub 4b2ebf5487 Ensure removed entities are not displayed in logbook (#37395) 2020-07-03 23:08:46 -07:00
J. Nick KostonandGitHub 7e664fbb3b Ensure logbook entries appear when the logbook.log (#37388)
service without a domain or entity_id
2020-07-02 19:53:28 -05:00
J. Nick KostonandGitHub a87c29b5d9 Ensure logbook performs well when filtering is configured (#37292) 2020-07-02 09:12:27 -07:00
J. Nick KostonandGitHub 76db2b39b0 Move logbook continuous domain filtering to sql (#37115)
* Move logbook continuous domain filtering to sql

sensors tend to generate a significant amount of states
that are filtered out by logbook.  In testing 75% of
states can be filtered away in sql to avoid the
sqlalchemy ORM overhead of creating objects that will
be discarded.

* remove un-needed nesting
2020-06-26 09:12:50 -05:00
Paulus SchoutsenandGitHub 5bc6ed4cef Add logbook platforms (#37078)
* Add logbook platforms

* Fix logbook describe test
2020-06-24 18:14:50 -07:00
mdegat01andGitHub 6c7355785a Add support for glob matching to entity filters (#36913)
* Added GLOB capability to entityfilter and every place that uses it. All existing tests are passing

* added tests for components affected by glob change

* fixed flake8 error

* mocking the correct listener

* mocking correct bus method in azure test

* tests passing in 3.7 and 3.8

* fixed formatting issue from rebase/conflict

* Checking against glob patterns in more performant way

* perf improvments and reverted unnecessarily adjusted tests

* added new benchmark test around filters

* no longer using get with default in entityfilter

* changed filter name and removed logbook from filter benchmark

* simplified benchmark tests from feedback

* fixed apache tests and returned include exclude schemas to normal

* fixed azure event hub tests to properly go through component logic

* fixed azure test and clean up for other tests

* renaming test files to match standard

* merged mqtt statestream test changes with base

* removed dependency on recorder filter schema from history

* fixed recorder tests after merge and a bunch of lint errors
2020-06-23 20:02:29 -05:00
J. Nick KostonandGitHub 5b79c1f9ef Remove support for deprecated hidden attribute from logbook and history (#37039) 2020-06-23 13:40:39 -07:00
J. Nick KostonandGitHub 53a91ece4e Improve isoformat timestamp performance (#36991)
* adj

* time_fired_isoformat

* remove unused code

* tests for processing timestamps

* restore missing import lost in merge conflict

* test for None case
2020-06-22 12:06:02 -05:00
J. Nick KostonandGitHub 1de97e3a35 Fix logbook filtering by entity id (#36973)
* Fix logbook filtering by entity_id

* remove debug
2020-06-21 14:34:47 -05:00
J. Nick KostonandGitHub 59e43ab6e4 Prefilter more logbook events in sql (#36958)
* Prefilter more logbook events in sql

Prefilter sensor events in _keep_event before humanify

Cache static attribute lookup

Reduces logbook execution time by ~35%

* fix mocking in benchmark

* Update tests for logbook users
2020-06-21 10:50:58 -07:00
J. Nick KostonandGitHub e7d982ee11 Improve db performance of state change events and reduce overall db size (#36883)
* Add old_state_id to states, remove old/new state data from events since it can now be found by a join

* remove state lookup on restart

* Ensure old_state is set for exisitng states
2020-06-17 22:26:41 -05:00
J. Nick KostonandGitHub b8c9f67533 Remove context_id from logbook api (#36796)
Reduces size of api response, execution, and download time.
2020-06-15 16:13:07 -05:00
J. Nick KostonandGitHub 0a219081ea Only process logbook timestamps for events we will keep (#36792)
* Only process logbook timestamps for events we will keep

Since we group by minute we were previously processing
every timestamp. We can avoid this by making all the
minute checks use the unprocessed datetime since
the groupings will be the same regardless of timezone.

This reduces the number of datetime object recreations
by at least an order of magnitude.
2020-06-15 14:54:20 -05:00
J. Nick KostonandGitHub b0163b65c6 Use states to avoid decoding logbook state changed events. (#36768)
avg 4.43s -> 1.88s
2020-06-15 13:53:05 -05:00
J. Nick KostonandGitHub e443dc1274 Speed up logbook with a lazy event decoder (#36730) 2020-06-15 10:53:38 -07:00
Franck NijhofandGitHub 238430136e Migrate script to use describe_event for logbook (#36729) 2020-06-12 16:44:29 -07:00
Franck NijhofandGitHub bf95658e24 Fix logbook filtering for described events (#36727) 2020-06-12 20:45:17 +02:00
Paulus SchoutsenandGitHub c33edbe5bb Use TestCase.addCleanup (#36560) 2020-06-08 12:26:40 -07:00
J. Nick KostonandGitHub 751e2f4125 Extend logbook api to be able to fetch a slice of data between two time boundaries. (#35847)
* Create logbook_timeslice api

* add end_datetime

* Add tests

* switch to end_time to match history api
2020-06-03 14:07:56 -05:00
J. Nick KostonandGitHub 578d4a9b6a Make the frontend available sooner (Part 1 of 2) (#36263)
* Part 1 of 2 (no breaking changes in part 1).

When integrations configured via the UI block startup or fail to start,
the webserver can remain offline which make it is impossible
to recover without manually changing files in
.storage since the UI is not available.

This change is the foundation that part 2 will build on
and enable a listener to start the webserver when the frontend
is finished loading.

Frontend Changes (home-assistant/frontend#6068)

* Address review comments

* bump timeout to 1800s, adjust comment

* bump timeout to 4h

* remove timeout failsafe

* and the test
2020-06-02 13:54:11 -05:00
Franck NijhofandGitHub e6fe34e64d Migrate automation to use describe_event for logbook (#36356) 2020-06-01 17:18:40 -07:00
Pascal VizeliandGitHub ed014e3c96 Revert "Ensure frontend is available if integrations fail to start - Part 1 of 2 (#36093)" (#36251)
This reverts commit fbe7b4ddfa.
2020-05-29 10:18:39 +02:00
fbe7b4ddfa Ensure frontend is available if integrations fail to start - Part 1 of 2 (#36093)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2020-05-28 21:09:07 -07:00
Paulus SchoutsenandGitHub ec47216388 Use built-in test helpers on 3.8 (#34901) 2020-04-30 13:29:50 -07:00
Paulus SchoutsenandGitHub 19be31d13a Migrate HomeKit to use describe_event for logbook support (#34485) 2020-04-20 17:48:09 -07:00
Franck NijhofandGitHub 98a2efcbab Collection of random (mainly) test improvements (#33733) 2020-04-06 12:51:48 +02:00
Franck NijhofandGitHub 906385172a String formatting improvements for tests (#33663) 2020-04-05 00:26:08 +02:00
J. Nick KostonandGitHub 2360fd4141 Switch legacy calls to init_recorder_component using async_add… (#33185) 2020-03-23 11:27:45 -07:00
J. Nick KostonandGitHub 3a680bf7b7 Add a commit interval setting to recorder (#32596)
* Add a commit interval setting to recorder

* Make the default every 1s instead of immediate

* See attached py-spy flamegraphs for why 1s

* This avoids disk thrashing during event storms

* Make Home Assistant significantly more responsive on busy systems

* remove debug

* Add commit forces for tests that expect commits to be immediate

* Add commit forces for tests that expect commits to be immediate

* make sure _trigger_db_commit is in the right place (all effective "wait_recording_done" calls)

* De-duplicate wait_recording_done code
2020-03-09 17:43:26 -07:00
Anders MelchiorsenandGitHub 17215709e1 Simplify logbook duplicate handling (#32572) 2020-03-07 20:27:51 -08:00
Paulus SchoutsenandGitHub 8aea538662 Allow teaching logbook about events (#32444)
* Allow teaching logbook about events

* Use async_add_executor_job

* Fix tests
2020-03-05 11:55:50 -08:00