Commit Graph

1775 Commits

Author SHA1 Message Date
Jon Chambers
ad32555cc9 Drop the old push scheduler Redis singleton. 2021-02-11 14:01:11 -05:00
Ehren Kret
be8a1acca9 Remove message database from the codebase (#395)
* Remove message database from the codebase

* Remove unused ExperimentEnrollmentManager in test

* Be more stylish
2021-02-11 10:50:03 -06:00
Jon Chambers
e0ed8fa0b8 Introduce a hyper-log-log-based cardinality rate limiter 2021-02-11 10:36:26 -05:00
Jon Chambers
dcbf285fae Un-ignore message controller tests. 2021-02-11 10:14:48 -05:00
Jon Chambers
71510a8199 Decommission the old directory cache. 2021-02-10 11:02:11 -05:00
Jon Chambers
2f105ed0a4 Add support for remote client deprecation 2021-02-09 12:24:44 -05:00
Jon Chambers
6c0de89de8 Remove scary message-throttling feature flags. 2021-02-08 11:55:01 -05:00
Jon Chambers
aa99e202b4 Clarify behavioral contract of the pre-key store 2021-02-08 11:45:57 -05:00
Jon Chambers
04728ea4bc Drop the old Postgres-based pre-key store. 2021-02-08 11:40:43 -05:00
Ehren Kret
d09b36b1d5 Fix deprecated imports for Java6Assertions and Matchers 2021-02-08 09:45:33 -06:00
Ehren Kret
a5dd4f5fac Remove signaling key from Device and AccountAttributes 2021-02-08 09:45:33 -06:00
Jon Chambers
3298db8683 Split traffic between the Postgres and Dynamo pre-key stores. 2021-02-04 15:18:53 -05:00
Jon Chambers
d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Moxie Marlinspike
5e1a572bd8 Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret
0dcb4b645c Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers
1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers
408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers
35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike
92f6a79e1f Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers
8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers
92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers
24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers
f10be893ce Drop the old feature flag controller. 2021-01-25 14:55:57 -05:00
Jon Chambers
c606c1664f Add admin tasks for listing, setting, and deleting feature flags. 2021-01-25 14:55:57 -05:00
Jon Chambers
225932b4c9 Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers
6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers
943a5d1036 Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Jon Chambers
0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers
a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Jon Chambers
81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00
Jon Chambers
9c53d818f4 Use separate clusters for message cache read/write operations. 2021-01-15 13:51:39 -05:00
Jon Chambers
e5a2c1ab10 Always return an empty list of prekeys. 2021-01-15 12:27:10 -05:00
Jon Chambers
67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Jon Chambers
2e01da5ec1 Add a task to enable/disable accelerated crawling. 2021-01-11 19:29:18 -05:00
Jon Chambers
9412a7424c Return HTTP/429 whenever somebody tries to get contacts from the old directory system. 2021-01-11 19:29:10 -05:00
Jon Chambers
f8cbb4f386 Temporarily suspend client version metrics to reduce load on our metric aggregator. 2021-01-11 14:04:44 -05:00
Ehren Kret
86ccaa52a5 Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00
Jon Chambers
1b53f10091 Reload scripts across the whole cluster if one shard is missing the script. 2021-01-10 15:00:12 -05:00
Jon Chambers
bac268a21c Don't send a reply to clients until messages are safely in a non-volatile store. 2021-01-10 13:03:40 -05:00
Moxie Marlinspike
22ef058cb6 Temporarily disable registration abuse system 2021-01-09 15:57:55 -05:00
Jon Chambers
9ee6419bc0 Publish directory updates to multiple SQS queues. 2021-01-08 18:07:18 -05:00
Jon Chambers
3bf0188e7f Turn off alphanumeric sender ID for all countries. 2021-01-08 06:18:53 -05:00
Jon Chambers
91fc0fd623 Revert "Delete data in the storage service when deleting accounts."
This reverts commit ff1a721d5b.
2021-01-08 06:18:39 -05:00
Jon Chambers
a4d0c17efd Record OS versions for iOS requests. 2020-12-23 11:36:31 -05:00
Jon Chambers
ff1a721d5b Delete data in the storage service when deleting accounts. 2020-12-23 11:35:38 -05:00
Jon Chambers
c870a1bbd5 Introduce a storage service client. 2020-12-23 11:35:38 -05:00
Jon Chambers
e551fd2c1b Revert "Pause checks for GV1 migration when checking for capability downgrades."
This reverts commit e7745db36e.
2020-12-10 17:02:41 -05:00
Jon Chambers
34a11c2338 Record OS versions for desktop and SDK versions for Android. 2020-12-10 17:02:05 -05:00
Jon Chambers
e524ff965d Add a utility method for getting client platform tags from UA strings for metrics. 2020-12-10 17:01:46 -05:00
Jon Chambers
92fde83b3a Discard oversized messages bound for desktop clients via websockets. 2020-12-07 15:03:35 -05:00