Commit Graph

167 Commits

Author SHA1 Message Date
Daniel Gasienica 082ef98a56 Add type definition for deferredToPromise 2018-04-25 15:24:51 -04:00
Daniel Gasienica 9d84b2f420 Index messages with attachments using a boolean
When indexing message attachment metadata using numeric indexes such as:

```javascript
{
  conversationId: '+12223334455',
  received_at: 123,
  attachments: […],
  numAttachments: 2,
},
{
  conversationId: '+12223334455',
  received_at: 456,
  attachments: [],
  numAttachments: 0,
}
{
  conversationId: '+12223334455',
  received_at: 789,
  attachments: [],
  numAttachments: 1,
}
```

It creates an index as follows:

```
[conversationId, received_at, numAttachments]
['+12223334455', 123, 2]
['+12223334455', 456, 0]
['+12223334455', 789, 1]
```

This means a query such as…

```
lowerBound: ['+12223334455', 0,                1               ]
upperBound: ['+12223334455', Number.MAX_VALUE, Number.MAX_VALUE]
```

…will return all three original entries because they span the `received_at`
from `0` through `Number.MAX_VALUE`. One workaround is to index booleans using
`1 | undefined` where `1` is included in the index and `undefined` is not, but
that way we lose the ability to query for the `false` value. Instead, we flip
adjust the index to `[conversationId, hasAttachments, received_at]` and can
then query messages with attachments using

```
[conversationId, 1 /* hasAttachments */, 0                /* received_at */]
[conversationId, 1 /* hasAttachments */, Number.MAX_VALUE /* received_at */]
```
2018-04-25 15:24:51 -04:00
Daniel Gasienica 5f220a7b2c Add migration for media gallery indices 2018-04-25 15:24:50 -04:00
Daniel Gasienica d14761087a Add schema version 4: Attachment metadata 2018-04-25 15:24:50 -04:00
Daniel Gasienica df2e6e7864 Port MIME module to TypeScript 2018-04-25 15:24:50 -04:00
Daniel Gasienica 6a63e427c8 Use is instead of Lodash is* 2018-04-25 15:24:50 -04:00
Scott Nonnenberg bf81c3db63 Document Message v4, remove data-free thumbnails 2018-04-23 16:43:25 -07:00
Scott Nonnenberg c02860af5c Responding to pull request review feedback
- messages.getQuoteObjectUrl: early return
- backup.js: explaining variables for long if statement
- types/messages.js: Log if thumbnail has neither data nor path
- sendmessage.js:
  - remove extraneous logging
  - fix indentation
  - upload attachments and thumbnails in parallel
- preload: don't load fs for tests, just fse
- _conversation.scss: split two selectors into two lines, 0px -> 0
- backup_test.js: use fse.existsSync and comment twoSlashes regex
- network_tests_view_test.js: Comment duplicate assignment to window.getSocketStatus
2018-04-23 15:36:47 -07:00
Scott Nonnenberg a7d44d3344 Backup and end-to-end test! 2018-04-20 15:24:04 -07:00
Scott Nonnenberg 978ff0e2bb Refactor: Use correct name for sanitized filename in backup.js 2018-04-20 15:24:02 -07:00
Scott Nonnenberg f1ff040842 Finish type definition for _mapQuotedAttachments 2018-04-16 12:16:34 -07:00
Scott Nonnenberg 9ad55c803f Fix handling attachment thumbnails using thumbnail key 2018-04-13 18:10:51 -07:00
Scott Nonnenberg 054d3887a1 Quotes: The full pipeline into the database
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
   disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
   destined for the database
2018-04-13 18:05:45 -07:00
Scott Nonnenberg 21bf02c94d Fixed examples in Quote.md, rough Android visuals 2018-04-13 18:05:44 -07:00
Daniel Gasienica d7b845326d ESLint auto-fix link_text 2018-04-11 16:36:42 -04:00
Daniel Gasienica 9d41b86162 Remove escaping from linkText
We leverage jQuery’s HTML escaping in `$.html(…)`.
2018-04-11 16:36:42 -04:00
Daniel Gasienica f04c65088b Fork link-text module 2018-04-11 16:36:42 -04:00
Daniel Gasienica 24f4ad53bc Use single quotes for identifiers 2018-04-11 15:54:32 -04:00
Daniel Gasienica 432a6ebd7f Redact file paths with escaped slashes 2018-04-11 15:30:59 -04:00
Daniel Gasienica a8a7525609 Redact stack traces with forward and backslashes 2018-04-11 15:30:59 -04:00
Daniel Gasienica b0da7d965e Redact URL encoded file paths in stack traces 2018-04-11 15:30:59 -04:00
Daniel Gasienica f2c9ccae90 Prefer is.* over Lodash is* functions 2018-04-11 15:30:59 -04:00
Daniel Gasienica 2ee78ec556 Prefer path over Path 2018-04-11 15:30:59 -04:00
Scott Nonnenberg c6c3b65bbc Introduce React, TypeScript, TSLint and React-StyleGuidist (#2219)
Quite a bit of change here.

First, the basics:

- New dependencies were added: react, typescript, tslint, and react-styleguidist
- A new npm script: transpile. It uses typescript to process .tsx files in js/react, putting .js files next to the original file. It's part of the watch functionality of grunt dev as well as the default task run with just grunt (used to build the app prior to release). A lighter-weight to get watch behavior when just working on React components is to run yarn transpile --watch.
- yarn run clean-transpile will remove generated .js files


Style guide via react-styleguidist. Example site: https://react-styleguidist.js.org/examples/basic/

- Start with yarn styleguide
- Component.md files right next to the .tsx file
- jsdoc-style comments are picked up and added to the generated part of the styleguide - the overall summary and a table listing methods and properties of the component
- It has hot-reloading!
- It uses webpack, which means that our app now pulls in webpack though we don't use it to generate anything for the production app.
- I did a bunch of work to enable the use of Backbone views in this context, which will allow us to move smoothly from the old world to the new. First, add all the permutations in the old way, and then slowly start to re-render those same views with React.

A bit of dependency cleanup to enable use in React components:

- moment was moved from our Bower dependencies to our npm dependencies, so it can be used in React components not running in a browser window.
- i18n was moved into the new commonjs format, so it can be used in React components even if window is not available.

Lastly, a bit of Gruntfile cleanup:

- Removal of Chrome App-era modifications of background.js
- Make jshint/jscs watch more targeted, since more and more we'll be using other tools
2018-04-06 08:13:00 -07:00
Scott Nonnenberg 50d4dbaae7 Remove 'use strict' from i18n.js 2018-04-05 13:42:41 -07:00
Scott Nonnenberg 76be13552f js/i18n.js -> js/modules/i18n.js 2018-04-05 12:51:29 -07:00
Daniel Gasienica 195e9b777c Rename willHaveAttachmentsSavedOnFileSystemDuringUpgrade
New name: `willAttachmentsGoToFileSystemOnUpgrade`
2018-04-04 19:11:46 -04:00
Daniel Gasienica 4c4443390a Rename createImporter to createAttachmentDataWriter 2018-04-04 19:11:46 -04:00
Daniel Gasienica 5ea1a305a6 Clarify error origin 2018-04-04 18:41:50 -04:00
Daniel Gasienica 9dcb64b53c Initialize schema version upon import 2018-04-04 18:41:50 -04:00
Daniel Gasienica cfad983d08 Consistently use ciphertext instead of encrypted 2018-04-04 18:41:50 -04:00
Daniel Gasienica a61c905146 Impliclity decrypt data if key is present 2018-04-04 18:41:50 -04:00
Daniel Gasienica 94c75762b5 Add Message.createAttachmentLoader 2018-04-04 18:41:50 -04:00
Daniel Gasienica 43ab320d31 Load attachments before writing them upon export 2018-04-04 18:41:50 -04:00
Daniel Gasienica 7eeabb5b1a Add precondition for writeAttachment 2018-04-04 18:41:50 -04:00
Daniel Gasienica 0faaf52b71 Import and upgrade messages upon save 2018-04-04 18:41:50 -04:00
Daniel Gasienica cf7355bd2f Rename backupToDirectory to exportToDirectory
For symmetry reasons.
2018-04-04 18:41:50 -04:00
Daniel Gasienica bf67254cc5 Add Message.createImporter 2018-04-04 18:41:50 -04:00
Daniel Gasienica efd673083d Rename writeAttachmentData to writeNewAttachmentData 2018-04-04 18:41:50 -04:00
Daniel Gasienica ce8fe0d345 Add Signal.Startup module 2018-04-04 18:10:52 -04:00
Daniel Gasienica 8474f3cf7f Use create* prefix to clarify curried functions 2018-04-03 15:25:24 -04:00
Daniel Gasienica 12cdeee7ec Rename count to numMessagesPerBatch 2018-04-03 14:43:17 -04:00
Daniel Gasienica 38ac9972e8 Improve termination condition 2018-04-03 13:16:05 -04:00
Daniel Gasienica 5b1a0c45df Replace getAll with cursor
This reduces the likelihood of IPC message size error.
2018-04-03 13:16:05 -04:00
Daniel Gasienica ed77378293 Fix bug in termination condition 2018-04-03 12:30:13 -04:00
Daniel Gasienica 052fb841f6 Allow database name override in migrations 2018-04-03 11:45:10 -04:00
Daniel Gasienica 1e04083813 Support database upgrades in Database.open 2018-04-03 11:44:50 -04:00
Daniel Gasienica f50ac1c99e Reduce excessive logging about database status 2018-04-03 11:06:56 -04:00
Daniel Gasienica 77b1f3c505 Replace startup with attachment background migration
Iterates through `Message::id` index.
2018-04-02 20:07:07 -04:00
Daniel Gasienica 057762806e Add support for attachment background migration without index 2018-04-02 19:41:28 -04:00