WIP version 13.0.0 (#3898)

* RENAME the `ajv-validate` plugin to `validate-ajv` to be in equal with the other validation plugins.

* FIX wrong filenames

* REFACTORED the validation plugins

* FIX ci

* FIX ci

* FIX ci

* FIX tests

* ADD run test suite once with validation

* FIX naming

* REFACTORED the [key compression plugin](https://rxdb.info/key-compression.html), it is no longer a plugin but now a wrapper around any other RxStorage.

* FIX tests

* FIX lint

* REFACTORED the encryption plugin, it is no longer a plugin but now a wrapper around any other RxStorage

* REMOVED many unused plugin hooks because they decreased the performance.

* FIX tests

* FIX pouchdb testss

* FIX loki tests

* REMOVE logs

* FIX node tests

* ADD debug for loki failure

* FIX lokijs tests

* REMOVED support for temporary documents

* FIX lint

* REMOVED RxDatabase.broadcastChannel

* FIX tests

* FIX test

* FIX angular example

* ADD debug log for loki

* ADD logs

* aDD log

* MOVE test

* CHANGE karma use better reporter

* FIX karma config

* FIX lokijs test

* fix react-native expo example

* rxdb installs from current branch

* ADD hint for mafintosh/is-my-json-valid#192

* CHANGE In RxDB `_meta.lwt` field, we now use 2 decimal number of the unix timestamp in milliseconds

* REPLACE is-my-json-valid with ajv

* REMOVE ajv strict mode

* FIX svelte example

* REFACTOR revision handling step 1

* CHANGE store the password hash in the same write request as the database token to improve performance

* FIX tests

* FIX tests

* FIX import

* REMOVE RxStorageStatics `.hash` and `.hashKey`

* FIX tests

* FIX tests

* FIX tests

* FIX tests

* FIX lint

* CHANGE removed default usage of `md5` as default hashing. Use a faster non-cryptographic hash instead

* FIX deps

* CHANGE use new replication protocol step 1

* FIX stuff for new replication

* FIX more replication tests

* FIX more graphql tests

* FIX more graphql tests

* FIX more graphql tests

* FIX pouchdb test

* FIX pouchdb tset

* FIX dexie.js tests

* FIX some more tests

* FIX ci

* ADD(docs) new replication protocol

* REMOVE liveInterval option from replication

* ADD logs for randomly failing test

* ADD logs for randomly failing test 2

* ADD logs for randomly failing test 3

* ADD logs for randomly failing test 4

* ADD logs for randomly failing test 5

* CHANGE use `Float` instead of `Int` to represent timestamps in GraphQL

* REMOVED support for the `deletedFlag` in the GraphQL replication. Use a [GraphQL alias](https://devinschulz.com/rename-fields-by-using-aliases-in-graphql/) instead.

* REPLACED `RxReplicationPullError` and `RxReplicationPushError` with normal `RxError` like in the rest of the RxDB code

* REMOVED the option to filter out replication documents with the push/pull modifiers [#2552](#2552) because this does not work with the new replication protocol

* FIX handle schema validation errors on replication

* REFACTOR graphql replication plugin

* ADD more docs for replication

* ADD docs

* CHANGE default of replication `live` to be set to `true`. Because most people want to do a live replication, not a one time replication.

* ADD(graphql-replication) pull.stream$

* FIX types

* FIX closing of sockets

* FIX import of isomorphic-ws

* FIX deps check

* FIX graphql query builders

* FIX graphql example step 1

* FIX graphql stream replication

* FIX graphql example

* FIX lint

* FIX angular example

* FIX deps check

* RENAME bulkSize to batchSize

* ADD deletedFlag

* FIX tests

* FIX tests

* FIX handling of deletedField

* CHANGED Attachment data is now always handled as `Blob` because Node.js does support `Blob` since version 18.0.0 so we no longer have to use a `Buffer` but instead can use Blob for browsers and Node.js

* FIX types

* FIX base64 handling of attachments data

* FIX pouchdb attachment handling

* FIX deleted field can be truthy

* IMPROVE logs

* FINISH docs for graphql replication

* FIX electron example

* FIX electron remote example

* FIX install script

* FIX tests

Co-authored-by: msotnikov <mishichka@gmail.com>

.github/workflows/main.yml

@@ -176,6 +176,10 @@ jobs:
         with:
           run: npm run test:browser:ci:memory
 
+      - name: npm run test:fast:memory-validation
+        run: npm run test:fast:memory-validation
+
+
   # run the node tests for the LokiJS RxStorage in a different
   # task to run in parallel.
   node-dexie-worker:
@@ -389,6 +393,7 @@ jobs:
         run: |
           npm install --legacy-peer-deps
           npm run build
+
       - name: electron install
         working-directory: ./examples/electron
         run: |
@@ -404,6 +409,7 @@ jobs:
       - name: electron-remote install
         working-directory: ./examples/electron-remote
         run: |
+          npm run preinstall
           npm install --legacy-peer-deps
 
       - name: electron-remote test

CHANGELOG.md

@@ -2,6 +2,41 @@
 # RxDB Changelog
 
 <!-- CHANGELOG NEWEST -->
+- RENAME the `ajv-validate` plugin to `validate-ajv` to be in equal with the other validation plugins.
+- The `is-my-json-valid` validation is no longer supported until [this bug](https://github.com/mafintosh/is-my-json-valid/pull/192) is fixed.
+- REFACTORED the [schema validation plugins](https://rxdb.info/schema-validation.html), they are no longer plugins but now they get wrapped around any other RxStorage.
+  - It allows us to run the validation inside of a [Worker RxStorage](./rx-storage-worker.md) instead of running it in the main JavaScript process.
+  - It allows us to configure which `RxDatabase` instance must use the validation and which does not. In production it often makes sense to validate user data, but you might not need the validation for data that is only replicated from the backend.
+- REFACTORED the [key compression plugin](https://rxdb.info/key-compression.html), it is no longer a plugin but now a wrapper around any other RxStorage.
+  - It allows to run the key-comresion inside of a [Worker RxStorage](./rx-storage-worker.md) instead of running it in the main JavaScript process.
+
+- REFACTORED the encryption plugin, it is no longer a plugin but now a wrapper around any other RxStorage.
+  - It allows to run the encryption inside of a [Worker RxStorage](./rx-storage-worker.md) instead of running it in the main JavaScript process.
+  - It allows do use asynchronous crypto function like [WebCrypto](https://developer.mozilla.org/en-US/docs/Web/API/Web_Crypto_API)
+- Store the password hash in the same write request as the database token to improve performance.
+
+- REMOVED many unused plugin hooks because they decreased the performance.
+
+- REMOVED support for temporary documents [see here](https://github.com/pubkey/rxdb/pull/3777#issuecomment-1120669088)
+- REMOVED RxDatabase.broadcastChannel The broadcast channel has been moved out of the RxDatabase and is part of the RxStorage. So it is not longer exposed via `RxDatabase.broadcastChannel`.
+
+- In the RxDB internal `_meta.lwt` field, we now use 2 decimals number of the unix timestamp in milliseconds.
+
+
+- REMOVE RxStorageStatics `.hash` and `.hashKey`
+
+- CHANGE removed default usage of `md5` as default hashing. Use a faster non-cryptographic hash instead.
+  - ADD option to pass a custom hash function when calling `createRxDatabase`.
+
+- Removed the `liveInterval` option of the replication. It was an edge case feature with wrong defaults. If you want to run the pull replication on internval, you can send a `RESYNC` event manually in a loop.
+
+- CHANGE use `Float` instead of `Int` to represent timestamps in GraphQL.
+
+- REPLACED `RxReplicationPullError` and `RxReplicationPushError` with normal `RxError` like in the rest of the RxDB code.
+- REMOVED the option to filter out replication documents with the push/pull modifiers [#2552](https://github.com/pubkey/rxdb/issues/2552) because this does not work with the new replication protocol.
+- CHANGE default of replication `live` to be set to `true`. Because most people want to do a live replication, not a one time replication.
+
+- CHANGED Attachment data is now always handled as `Blob` because Node.js does support `Blob` since version 18.0.0 so we no longer have to use a `Buffer` but instead can use Blob for browsers and Node.js
 
 <!-- ADD new changes here! -->
 

config/karma.conf.js

@@ -63,7 +63,8 @@ const configuration = {
         'karma-firefox-launcher',
         'karma-ie-launcher',
         'karma-opera-launcher',
-        'karma-detect-browsers'
+        'karma-detect-browsers',
+        'karma-spec-reporter'
     ],
 
     // Source files that you wanna generate coverage for.
@@ -86,7 +87,15 @@ const configuration = {
     },
     browserDisconnectTimeout: 12000,
     processKillTimeout: 12000,
-    singleRun: true
+    singleRun: true,
+
+
+    /**
+     * Use this reported to fully log all test names
+     * which makes it easier to debug.
+     * @link https://github.com/tmcgee123/karma-spec-reporter
+     */
+    reporters: ['spec']
 };
 
 if (process.env.CI) {

docs-src/SUMMARY.md

@@ -54,7 +54,6 @@
         * [insert()](./rx-collection.md#insert)
         * [bulkInsert()](./rx-collection.md#bulkinsert)
         * [bulkRemove()](./rx-collection.md#bulkremove)
-        * [newDocument()](./rx-collection.md#newdocument)
         * [upsert()](./rx-collection.md#upsert)
         * [atomicUpsert()](./rx-collection.md#atomicupsert)
         * [find()](./rx-collection.md#find)
@@ -123,9 +122,9 @@
     * [RxStorage Memory Synced](./rx-storage-memory-synced.md)
     * [RxStorage Sharding](./rx-storage-sharding.md)
 
-* [Replication CouchDB](./replication-couchdb.md)
-* [Replication GraphQL](./replication-graphql.md)
-* [Replication Primitives](./replication.md)
+* [Replication](./replication.md)
+    * [Replication GraphQL](./replication-graphql.md)
+    * [Replication CouchDB](./replication-couchdb.md)
 
 * [Cleanup](./cleanup.md)