Commit Graph
55 Commits
Author SHA1 Message Date
Daniel Gasienica f36f206a01 Use IndexablePresence for hasFileAttachments and hasVisualMediaAttachments
Reduces index size, makes it easier to debug using IndexedDB inspector, and
hopefully improves lookup performance.
2018-04-25 15:25:12 -04:00
Daniel Gasienica 8e3c38d5fe Fix formatting 2018-04-25 15:25:12 -04:00
Daniel Gasienica 146178f977 Update schema documentation 2018-04-25 15:24:51 -04:00
Daniel Gasienica 9d84b2f420 Index messages with attachments using a boolean
When indexing message attachment metadata using numeric indexes such as:

```javascript
{
  conversationId: '+12223334455',
  received_at: 123,
  attachments: […],
  numAttachments: 2,
},
{
  conversationId: '+12223334455',
  received_at: 456,
  attachments: [],
  numAttachments: 0,
}
{
  conversationId: '+12223334455',
  received_at: 789,
  attachments: [],
  numAttachments: 1,
}
```

It creates an index as follows:

```
[conversationId, received_at, numAttachments]
['+12223334455', 123, 2]
['+12223334455', 456, 0]
['+12223334455', 789, 1]
```

This means a query such as…

```
lowerBound: ['+12223334455', 0,                1               ]
upperBound: ['+12223334455', Number.MAX_VALUE, Number.MAX_VALUE]
```

…will return all three original entries because they span the `received_at`
from `0` through `Number.MAX_VALUE`. One workaround is to index booleans using
`1 | undefined` where `1` is included in the index and `undefined` is not, but
that way we lose the ability to query for the `false` value. Instead, we flip
adjust the index to `[conversationId, hasAttachments, received_at]` and can
then query messages with attachments using

```
[conversationId, 1 /* hasAttachments */, 0                /* received_at */]
[conversationId, 1 /* hasAttachments */, Number.MAX_VALUE /* received_at */]
```
2018-04-25 15:24:51 -04:00
Daniel Gasienica d14761087a Add schema version 4: Attachment metadata 2018-04-25 15:24:50 -04:00
Daniel Gasienica df2e6e7864 Port MIME module to TypeScript 2018-04-25 15:24:50 -04:00
Daniel Gasienica 6a63e427c8 Use is instead of Lodash is* 2018-04-25 15:24:50 -04:00
Scott Nonnenberg bf81c3db63 Document Message v4, remove data-free thumbnails 2018-04-23 16:43:25 -07:00
Scott Nonnenberg c02860af5c Responding to pull request review feedback
- messages.getQuoteObjectUrl: early return
- backup.js: explaining variables for long if statement
- types/messages.js: Log if thumbnail has neither data nor path
- sendmessage.js:
  - remove extraneous logging
  - fix indentation
  - upload attachments and thumbnails in parallel
- preload: don't load fs for tests, just fse
- _conversation.scss: split two selectors into two lines, 0px -> 0
- backup_test.js: use fse.existsSync and comment twoSlashes regex
- network_tests_view_test.js: Comment duplicate assignment to window.getSocketStatus
2018-04-23 15:36:47 -07:00
Scott Nonnenberg a7d44d3344 Backup and end-to-end test! 2018-04-20 15:24:04 -07:00
Scott Nonnenberg f1ff040842 Finish type definition for _mapQuotedAttachments 2018-04-16 12:16:34 -07:00
Scott Nonnenberg 9ad55c803f Fix handling attachment thumbnails using thumbnail key 2018-04-13 18:10:51 -07:00
Scott Nonnenberg 054d3887a1 Quotes: The full pipeline into the database
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
   disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
   destined for the database
2018-04-13 18:05:45 -07:00
Scott Nonnenberg 21bf02c94d Fixed examples in Quote.md, rough Android visuals 2018-04-13 18:05:44 -07:00
Daniel Gasienica 24f4ad53bc Use single quotes for identifiers 2018-04-11 15:54:32 -04:00
Daniel Gasienica 195e9b777c Rename willHaveAttachmentsSavedOnFileSystemDuringUpgrade
New name: `willAttachmentsGoToFileSystemOnUpgrade`
2018-04-04 19:11:46 -04:00
Daniel Gasienica 4c4443390a Rename createImporter to createAttachmentDataWriter 2018-04-04 19:11:46 -04:00
Daniel Gasienica 5ea1a305a6 Clarify error origin 2018-04-04 18:41:50 -04:00
Daniel Gasienica 9dcb64b53c Initialize schema version upon import 2018-04-04 18:41:50 -04:00
Daniel Gasienica 94c75762b5 Add Message.createAttachmentLoader 2018-04-04 18:41:50 -04:00
Daniel Gasienica bf67254cc5 Add Message.createImporter 2018-04-04 18:41:50 -04:00
Daniel Gasienica efd673083d Rename writeAttachmentData to writeNewAttachmentData 2018-04-04 18:41:50 -04:00
Daniel Gasienica d9be6a0f94 Destructure Lodash requires 2018-04-02 15:12:04 -04:00
Daniel Gasienica 5910f84af4 Remove outdated documentation 2018-04-02 12:17:36 -04:00
Daniel Gasienica 4651cbc2eb Improve `Message.upgradeSchema preconditions 2018-03-26 16:49:21 -04:00
Daniel Gasienica 8e762797c0 Remove attachment validation
Turns out `fileName` is optional and maybe even `contentType`. We should revisit
this at a later point in time.
2018-03-26 16:49:21 -04:00
Daniel Gasienica 867f73b80a Use double quotes for identifiers in error messages 2018-03-26 16:38:34 -04:00
Daniel Gasienica 2ad0fd7db6 Use Lodash omit instead of delete 2018-03-26 16:38:34 -04:00
Daniel Gasienica 279b3f81c7 Add Attachment.deleteData 2018-03-26 16:38:34 -04:00
Daniel Gasienica 9a540d6d18 Load attachment data before sending message 2018-03-26 16:38:34 -04:00
Daniel Gasienica 6355c54114 Return relative path from writeAttachmentData
This will make our app more robust to changes in location on the file system.
2018-03-26 16:38:34 -04:00
Daniel Gasienica 2cd3d5ac16 Add Message schema version 3 2018-03-26 16:38:34 -04:00
Daniel Gasienica aa82a2a5fb Add context to Message.upgradeSchema
Allows us to pass in Electron/Node.js specific functions. This allows better
unit testing in Mocha where we don’t have access to Electron APIs.
2018-03-26 16:38:34 -04:00
Daniel Gasienica ebe2a769c9 Add Attachment.migrateDataToFileSystem 2018-03-26 16:38:34 -04:00
Daniel Gasienica a619d48fac Update schema version history 2018-03-19 19:27:59 -04:00
Daniel Gasienica 0e20e8e2ea Use Errors.toLogFormat 2018-03-19 19:27:59 -04:00
Daniel Gasienica a5edbf8328 Initialize schema as first step in upgradeSchema 2018-03-19 19:27:59 -04:00
Daniel Gasienica 752c8f97e6 🎨 Format ternaries 2018-03-19 19:27:59 -04:00
Daniel Gasienica 8dfaa5619f Prefix private functions with underscore 2018-03-19 19:27:59 -04:00
Daniel Gasienica ed336d31d7 Move schema versioning from Attachment to Message 2018-03-19 19:27:59 -04:00
Daniel Gasienica b9e9f5e19a 🎨 withInheritedSchemaVersion to inheritSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica df693ade7c Allow Attachment.fileName to be null 2018-03-19 19:27:59 -04:00
Daniel Gasienica c27746b79e Add Message.withInheritedSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica e9e46464c2 Add SchemaVersion type 2018-03-19 19:27:59 -04:00
Daniel Gasienica add4b11df3 Skip invalid attachments and make function sync 2018-03-19 19:27:59 -04:00
Daniel Gasienica 3dfc823716 Add Attachment.removeSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica 43b47fd349 Avoid ensure-error for privacy reasons
Example:

```
> node
> const nonError = {foo: 'i-am-private'};
undefined

// before
> util.inspect(nonError);
'{ foo: \'i-am-private\' }'

// after
> nonError.toString()
'[object Object]'
>
```
2018-03-08 16:23:51 -05:00
Daniel Gasienica 44b81f68dd Remove privacy redaction from Errors.toLogFormat 2018-03-08 16:23:51 -05:00
Daniel Gasienica 49e0850fb2 Extract Privacy module
Centralizes how we redact sensitive information.
2018-03-08 16:23:51 -05:00
Daniel Gasienica 0e2f8a8a06 Extract Errors.redactSensitivePaths 2018-03-08 16:22:38 -05:00