Commit Graph

56 Commits

Author SHA1 Message Date
Jon Chambers
ff448950ed Collapse the feature flag system into the dynamic config system. 2021-02-12 15:23:34 -05:00
Jon Chambers
e0ed8fa0b8 Introduce a hyper-log-log-based cardinality rate limiter 2021-02-11 10:36:26 -05:00
Jon Chambers
71510a8199 Decommission the old directory cache. 2021-02-10 11:02:11 -05:00
Jon Chambers
2f105ed0a4 Add support for remote client deprecation 2021-02-09 12:24:44 -05:00
Ehren Kret
6865cdfce3 Replace all org.hibernate validators with modern javax.validation 2021-02-08 09:45:33 -06:00
Jon Chambers
1dceee3fa0 Make DynamoDBConfiguration more consistent. 2021-02-05 10:40:27 -05:00
Jon Chambers
d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Moxie Marlinspike
5e1a572bd8 Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret
0dcb4b645c Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers
35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike
92f6a79e1f Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers
611e8c39ee Actually drop feature flag config. 2021-01-25 15:20:06 -05:00
Jon Chambers
67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Ehren Kret
86ccaa52a5 Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00
Jon Chambers
9ee6419bc0 Publish directory updates to multiple SQS queues. 2021-01-08 18:07:18 -05:00
Jon Chambers
c870a1bbd5 Introduce a storage service client. 2020-12-23 11:35:38 -05:00
Jon Chambers
6bb106c2cb Drop the Redis command timeout back down to 3 seconds to facilitate debug data collection. 2020-12-11 11:20:10 -05:00
Jon Chambers
1a627d6a87 Extend Redis command timeout to 3.5 seconds to avoid TCP retransmission "coincidences." 2020-11-25 15:04:06 -05:00
Jon Chambers
251e1b51c5 Make Micrometer batch size configurable. 2020-11-13 17:13:39 -05:00
Jon Chambers
b25e50bdae Drop API keys from Micrometer configuration. 2020-11-09 09:26:56 -05:00
Ehren Kret
604287244f Update copyright statement on all source files
IntelliJ Copyright Profile used to automate this.
2020-11-04 11:55:35 -05:00
Curt Brune
39c09733d3 Add /v1/payments/auth endpoint 2020-10-08 10:51:01 -04:00
Jon Chambers
e83b41dc01 Reduce default Redis cluster command timeout to 3 seconds. 2020-10-01 17:17:07 -04:00
Jon Chambers
76665dd56e Retry Redis commands that time out. 2020-10-01 17:17:07 -04:00
Jon Chambers
18ecd748dd Entirely discard the old message cache machinery. 2020-09-08 09:30:47 -04:00
Jon Chambers
ac465c5a18 Add a Lettuce-based Redis singleton client. 2020-08-27 10:51:12 -04:00
Jon Chambers
1ef3546822 Add support for server-side feature flags 2020-08-26 20:27:33 -04:00
Jon Chambers
a709a3bcc0 Remove a candidate metric provider. 2020-08-20 15:40:56 -04:00
Jon Chambers
df90de3a5f Change default Lettuce command timeout to 10s. 2020-08-18 16:21:42 -04:00
Jon Chambers
42ea7a9814 Revert Lettuce connection pooling. 2020-08-18 16:21:42 -04:00
Jon Chambers
d243b73678 Make Lettuce connection pools configurable. Double the default size. 2020-08-18 12:20:12 -04:00
Jon Chambers
27f721a1f5 Update to resilience4j 1.5.0. 2020-08-14 17:15:56 -04:00
Ehren Kret
b97158bf7b Create global remote config controllable in the signal server configuration (#127)
* Add global config controller through file rather than database

* Do no permit attempting to set or delete global config entries
2020-08-10 16:31:15 -05:00
Ehren Kret
4fa3a136ad Remove arbitrary SMS and add a NANPA message service (#123)
* Remove arbitrary SMS code

This code has run its course and is no longer needed for now.

* Add elements to sample config that were left out

* Add a messaging service for NANPA

* Fixup sample config capitalization
2020-08-05 13:35:11 -05:00
Ehren Kret
7cb24dd96d Add environment tag to datadog metric reporting 2020-07-30 18:04:16 -04:00
Jon Chambers
6fc1b4c6c0 Add a cluster-backed message cache. 2020-07-30 11:39:14 -04:00
Ehren Kret
89bafea61f Move SMS strings to configuration 2020-07-27 11:23:21 -05:00
Jon Chambers
33a0c4a9ae Use first party metric aggregator libraries where possible. 2020-07-24 17:21:56 -04:00
Jon Chambers
4cc5999f05 Configure additional metric aggregators. 2020-07-23 13:31:19 -04:00
Ehren Kret
b7e0e5a356 Create a strategy class to decide which sender id to use
The rules around selecting sender ids can get complicated with some
countries not supporting it and others requiring pre-registration that
may result in having a different sender id for that country than
others. This strategy class handles the logic of dealing with this
expanded configuration and applying the appropriate sender id or none
when it's not appropriate to do so at all.
2020-07-14 10:53:48 -05:00
Ehren Kret
6c1ba957bd Ensure the default alphaId configuration is an empty list rather than null 2020-07-07 10:17:40 -05:00
Ehren Kret
e021286eee Add configuration by country for sending from alpha IDs 2020-07-07 10:17:40 -05:00
Jon Chambers
ec858b2d4c Set a timeout for Redis cluster operations and shut down the cluster as part of service shutdown 2020-06-07 18:27:57 -04:00
Jon Chambers
fe1054d58a Introduce a Lettuce-based fault-tolerant Redis cluster accessor. 2020-06-06 20:37:48 -04:00
Jon Chambers
45ad8f8ffb Add the Wavefront/Micrometer reporter as a dependency and configure a registry. 2020-05-20 17:46:07 -04:00
Jon Chambers
b1d11d4f69 Use APNs signing keys instead of expiring certificates. 2020-05-12 12:48:28 -04:00
Ehren Kret
41286650cc Create attachments V3 endpoint for CDN2 on GCP
In preparation for resumable uploads, this creates a separate
attachment authorization endpoint that creates a signed URL for
accessing GCP Storage through Signal's CDN2. This should allow Signal
clients to do byte-level resume of media uploads.
2020-04-02 10:20:42 -07:00
Moxie Marlinspike
ac1153c7cf Additional limits 2020-03-14 18:10:07 -07:00
Moxie Marlinspike
08a70664f4 Support for getting/setting remote config variables 2020-01-21 13:38:58 -08:00
Moxie Marlinspike
75fc35ee4b Parameterize access to zk operations 2020-01-21 11:29:08 -08:00