Commit Graph

55 Commits

Author SHA1 Message Date
Daniel Gasienica
f36f206a01 Use IndexablePresence for hasFileAttachments and hasVisualMediaAttachments
Reduces index size, makes it easier to debug using IndexedDB inspector, and
hopefully improves lookup performance.
2018-04-25 15:25:12 -04:00
Daniel Gasienica
8e3c38d5fe Fix formatting 2018-04-25 15:25:12 -04:00
Daniel Gasienica
146178f977 Update schema documentation 2018-04-25 15:24:51 -04:00
Daniel Gasienica
9d84b2f420 Index messages with attachments using a boolean
When indexing message attachment metadata using numeric indexes such as:

```javascript
{
  conversationId: '+12223334455',
  received_at: 123,
  attachments: […],
  numAttachments: 2,
},
{
  conversationId: '+12223334455',
  received_at: 456,
  attachments: [],
  numAttachments: 0,
}
{
  conversationId: '+12223334455',
  received_at: 789,
  attachments: [],
  numAttachments: 1,
}
```

It creates an index as follows:

```
[conversationId, received_at, numAttachments]
['+12223334455', 123, 2]
['+12223334455', 456, 0]
['+12223334455', 789, 1]
```

This means a query such as…

```
lowerBound: ['+12223334455', 0,                1               ]
upperBound: ['+12223334455', Number.MAX_VALUE, Number.MAX_VALUE]
```

…will return all three original entries because they span the `received_at`
from `0` through `Number.MAX_VALUE`. One workaround is to index booleans using
`1 | undefined` where `1` is included in the index and `undefined` is not, but
that way we lose the ability to query for the `false` value. Instead, we flip
adjust the index to `[conversationId, hasAttachments, received_at]` and can
then query messages with attachments using

```
[conversationId, 1 /* hasAttachments */, 0                /* received_at */]
[conversationId, 1 /* hasAttachments */, Number.MAX_VALUE /* received_at */]
```
2018-04-25 15:24:51 -04:00
Daniel Gasienica
d14761087a Add schema version 4: Attachment metadata 2018-04-25 15:24:50 -04:00
Daniel Gasienica
df2e6e7864 Port MIME module to TypeScript 2018-04-25 15:24:50 -04:00
Daniel Gasienica
6a63e427c8 Use is instead of Lodash is* 2018-04-25 15:24:50 -04:00
Scott Nonnenberg
bf81c3db63 Document Message v4, remove data-free thumbnails 2018-04-23 16:43:25 -07:00
Scott Nonnenberg
c02860af5c Responding to pull request review feedback
- messages.getQuoteObjectUrl: early return
- backup.js: explaining variables for long if statement
- types/messages.js: Log if thumbnail has neither data nor path
- sendmessage.js:
  - remove extraneous logging
  - fix indentation
  - upload attachments and thumbnails in parallel
- preload: don't load fs for tests, just fse
- _conversation.scss: split two selectors into two lines, 0px -> 0
- backup_test.js: use fse.existsSync and comment twoSlashes regex
- network_tests_view_test.js: Comment duplicate assignment to window.getSocketStatus
2018-04-23 15:36:47 -07:00
Scott Nonnenberg
a7d44d3344 Backup and end-to-end test! 2018-04-20 15:24:04 -07:00
Scott Nonnenberg
f1ff040842 Finish type definition for _mapQuotedAttachments 2018-04-16 12:16:34 -07:00
Scott Nonnenberg
9ad55c803f Fix handling attachment thumbnails using thumbnail key 2018-04-13 18:10:51 -07:00
Scott Nonnenberg
054d3887a1 Quotes: The full pipeline into the database
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
   disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
   destined for the database
2018-04-13 18:05:45 -07:00
Scott Nonnenberg
21bf02c94d Fixed examples in Quote.md, rough Android visuals 2018-04-13 18:05:44 -07:00
Daniel Gasienica
24f4ad53bc Use single quotes for identifiers 2018-04-11 15:54:32 -04:00
Daniel Gasienica
195e9b777c Rename willHaveAttachmentsSavedOnFileSystemDuringUpgrade
New name: `willAttachmentsGoToFileSystemOnUpgrade`
2018-04-04 19:11:46 -04:00
Daniel Gasienica
4c4443390a Rename createImporter to createAttachmentDataWriter 2018-04-04 19:11:46 -04:00
Daniel Gasienica
5ea1a305a6 Clarify error origin 2018-04-04 18:41:50 -04:00
Daniel Gasienica
9dcb64b53c Initialize schema version upon import 2018-04-04 18:41:50 -04:00
Daniel Gasienica
94c75762b5 Add Message.createAttachmentLoader 2018-04-04 18:41:50 -04:00
Daniel Gasienica
bf67254cc5 Add Message.createImporter 2018-04-04 18:41:50 -04:00
Daniel Gasienica
efd673083d Rename writeAttachmentData to writeNewAttachmentData 2018-04-04 18:41:50 -04:00
Daniel Gasienica
d9be6a0f94 Destructure Lodash requires 2018-04-02 15:12:04 -04:00
Daniel Gasienica
5910f84af4 Remove outdated documentation 2018-04-02 12:17:36 -04:00
Daniel Gasienica
4651cbc2eb Improve `Message.upgradeSchema preconditions 2018-03-26 16:49:21 -04:00
Daniel Gasienica
8e762797c0 Remove attachment validation
Turns out `fileName` is optional and maybe even `contentType`. We should revisit
this at a later point in time.
2018-03-26 16:49:21 -04:00
Daniel Gasienica
867f73b80a Use double quotes for identifiers in error messages 2018-03-26 16:38:34 -04:00
Daniel Gasienica
2ad0fd7db6 Use Lodash omit instead of delete 2018-03-26 16:38:34 -04:00
Daniel Gasienica
279b3f81c7 Add Attachment.deleteData 2018-03-26 16:38:34 -04:00
Daniel Gasienica
9a540d6d18 Load attachment data before sending message 2018-03-26 16:38:34 -04:00
Daniel Gasienica
6355c54114 Return relative path from writeAttachmentData
This will make our app more robust to changes in location on the file system.
2018-03-26 16:38:34 -04:00
Daniel Gasienica
2cd3d5ac16 Add Message schema version 3 2018-03-26 16:38:34 -04:00
Daniel Gasienica
aa82a2a5fb Add context to Message.upgradeSchema
Allows us to pass in Electron/Node.js specific functions. This allows better
unit testing in Mocha where we don’t have access to Electron APIs.
2018-03-26 16:38:34 -04:00
Daniel Gasienica
ebe2a769c9 Add Attachment.migrateDataToFileSystem 2018-03-26 16:38:34 -04:00
Daniel Gasienica
a619d48fac Update schema version history 2018-03-19 19:27:59 -04:00
Daniel Gasienica
0e20e8e2ea Use Errors.toLogFormat 2018-03-19 19:27:59 -04:00
Daniel Gasienica
a5edbf8328 Initialize schema as first step in upgradeSchema 2018-03-19 19:27:59 -04:00
Daniel Gasienica
752c8f97e6 🎨 Format ternaries 2018-03-19 19:27:59 -04:00
Daniel Gasienica
8dfaa5619f Prefix private functions with underscore 2018-03-19 19:27:59 -04:00
Daniel Gasienica
ed336d31d7 Move schema versioning from Attachment to Message 2018-03-19 19:27:59 -04:00
Daniel Gasienica
b9e9f5e19a 🎨 withInheritedSchemaVersion to inheritSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
df693ade7c Allow Attachment.fileName to be null 2018-03-19 19:27:59 -04:00
Daniel Gasienica
c27746b79e Add Message.withInheritedSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
e9e46464c2 Add SchemaVersion type 2018-03-19 19:27:59 -04:00
Daniel Gasienica
add4b11df3 Skip invalid attachments and make function sync 2018-03-19 19:27:59 -04:00
Daniel Gasienica
3dfc823716 Add Attachment.removeSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
43b47fd349 Avoid ensure-error for privacy reasons
Example:

```
> node
> const nonError = {foo: 'i-am-private'};
undefined

// before
> util.inspect(nonError);
'{ foo: \'i-am-private\' }'

// after
> nonError.toString()
'[object Object]'
>
```
2018-03-08 16:23:51 -05:00
Daniel Gasienica
44b81f68dd Remove privacy redaction from Errors.toLogFormat 2018-03-08 16:23:51 -05:00
Daniel Gasienica
49e0850fb2 Extract Privacy module
Centralizes how we redact sensitive information.
2018-03-08 16:23:51 -05:00
Daniel Gasienica
0e2f8a8a06 Extract Errors.redactSensitivePaths 2018-03-08 16:22:38 -05:00