BREAKING CHANGE: The MongoDB aggregation pipeline requires native MongoDB syntax instead of the custom Parse Server syntax; for example pipeline stage names require a leading dollar sign like `$match` and the MongoDB document ID is referenced using `_id` instead of `objectId` (#8362)
BREAKING CHANGE: Nested objects are now properly stored in the database using JSON serialization; previously, due to a bug only top-level objects were serialized, but nested objects were saved as raw JSON; for example, a nested `Date` object was saved as a JSON object like `{ "__type": "Date", "iso": "2020-01-01T00:00:00.000Z" }` instead of its serialized representation `2020-01-01T00:00:00.000Z` (#8209)
BREAKING CHANGE: The MongoDB GridStore adapter has been removed. By default, Parse Server already uses GridFS, so if you do not manually use the GridStore adapter, you can ignore this change.
* Added a test case that triggers the query parameter crash
* rest.js: validate the explain parameter to keep the nodejs driver from throwing an uncatchable exception and crashing the server (see https://jira.mongodb.org/browse/NODE-3463)
RestQuery.js: Check whether explain mode is enabled not by "!== true", but by the "!" operator. explain can have string values.
Added tests that validate correct behaviour on different explain values
* Refactor the new tests
* Simplify the new tests
Also do a sanity check on the explain results
* Test refactor
* Exclude queryPlannerExtended as it is not supported by the testing environment
Simplifies the tests
* Restrict the changes to mongodb
Moved the verification of the explain value from rest.js to MongoStorageAdapter.js
Also restricted the relevant unit tests to mongodb
* Added changelog entry
* reformat changelog entry
* Update CHANGELOG.md
Co-authored-by: Kartal Kaan Bozdoğan <kartalkaanbozdogan@gmail.com>
Co-authored-by: Manuel <5673677+mtrezza@users.noreply.github.com>
* Fixed a bug affecting updates to nested pointers
Also created unit tests
* Marked the regression test for #7391 as pending for postgre
The issue is not fixed yet
Use cont instead of var
* Fix flaky test with transactions
* Add CHANGELOG entry
* Fix the other transactions related tests that became flaky because now Parse Server tries to submit the transaction multilpe times in the case of TransientError
* Remove fit from tests
* Update PostgresStorageAdapter.js
Improve `createClass` transaction:
* `await` makes it a more consistent sequence of queries
* `batch` is not needed there
* No need for an extra `.then` section
* Update PostgresStorageAdapter.js
Remove batch-dependent error code check, as it should happen automatically without batch result.
* Update PostgresStorageAdapter.js
Removing unused variable.
* prepend className to unique index to allow multiple unique indexes for different classes
* add testcase
* switched test so it can be tested on older versions of parse-server and show failure
* get rid of console log messages on restart by checking if the index exists before creating it
* add IF NOT EXISTS and IF EXISTS to ALTER TABLE
* revert some of code
* ensureIndex use IF NOT EXISTS
* ALTER TABLE CONSTRAINT can't use IF, ADD/DROP COLUMN can
* retesting
* update
* switchted to CREATE UNIQUE INDEX instrad of ALTER TABLE... ALTER TABLE doesn't seem to be needed
* Optimize query, fixes some null returns, fix stitched GraphQLUpload
* Fix authData key selection
* Prefer Iso string since other GraphQL solutions use this format
* fix tests
Co-authored-by: Antonio Davi Macedo Coelho de Castro <adavimacedo@gmail.com>
* add test cases for geoNear aggregation
Test cases do not have the `query` parameter set in $geoNear aggregation stage. this is to test for a reported potential issue when the parameter is not set.
* fixed potential issue when setting the geoNear.query parameter to undefined
see dicussion in https://github.com/parse-community/parse-server/pull/6540
* fixed duplicate index name in test
* use pg-promise native pg-connection-string to parse uri instead of ParseConfigParser.js. The allows for a more felxible uri for ssl and other params
* added ssl config params and others to PostgresConfigParser
* forgot to add back the original client file
* need to read in file at path for pfx, ca, key, and key
* convert file buffer to string to be consistant with node-postgres examples
* Fixing objectId for Pointer in Postgres
* add test case for longer objectId pointer. Note that this test fails on Postgres before the addition of previous commit
* removed comment that wasn't needed
* Apply linter changes on files I'm about to update
My actual changes were quite difficult to find when buried in this sea
of style changes, which were getting automatically applied during a
pre-commit hook. Here I just run the hooks against the files I'm going
to be touching in the following commit, so that a reviewer can ignore
these automatically generated diffs and just view the meaningful commit.
* perf: Allow covering relation queries with minimal index
When finding objects through a relation, we're sending Mongo queries
that look like this:
```
db.getCollection('_Join:foo:bar').find({ relatedId: { $in: [...] } });
```
From the result of that query, we're only reading the `owningId` field,
so we can start by adding it as a projection:
```
db.getCollection('_Join:foo:bar')
.find({ relatedId: { $in: [...] } })
.project({ owningId: 1 });
```
This seems like the perfect example of a query that could be satisfied
with an index scan: we are querying on one field, and only need one
field from the matching document.
For example, this can allow users to speed up the fetching of user roles
in authentication, because they query a `roles` relation on the `_Role`
collection. To add a covering index on that, you could now add an index
like the following:
```
db.getCollection('_Join:roles:_Role').createIndex(
{ relatedId: 1, owningId: 1 },
{ background: true }
);
```
One caveat there is that the index I propose above doesn't include the
`_id` column. For the query in question, we don't actually care about
the ID of the row in the join table, just the `owningId` field, so we
can avoid some overhead of putting the `_id` column into the index if we
can also drop it from the projection. This requires adding a small
special case to the MongoStorageAdapter, because the `_id` field is
special: you have to opt-out of using it by projecting `{ _id: 0 }`.
* Update .travis.yml
testing error to see what happens...
* Update .travis.yml
Attempting to resolve postgres in CL by installing postgis via sudo instead of through apt/packages
* Update .travis.yml
* Update .travis.yml
* Update .travis.yml
Removed extra lines of postgres that were under "services" and "addons". I believe the "postgresql" line under "services" was installing the default of 9.6 and "addons" was installing postgres 11. My guess is the fail was occurring due to 9.6 being called sometimes and it never had postgis installed. If this is true, the solution is to only install one version of postgres, which is version 11 with postgis 2.5.
* Adding test case for caseInsensitive
Adding test case for verifying indexing for caseInsensitive
* Implementing ensureIndex
* Updated PostgresStorageAdapter calls to ST_DistanceSphere. Note this has a minimum requirement of postgis 2.2. Documented the change in the readme. This is address #6441
* updated postgres sections of contributions with newer postgres info. Also switched postgis image it points to as the other one hasn't been updated in over a year.
* more info about postgres
* added necessary password for postgres docker
* updated wording in contributions
* removed reference to MacJr environment var when starting postgres in contributions. The official image automatically creates a user named 'postgres', but it does require a password, which the command sets to 'postgres'
* added more time to docker sleep/wait to enter postgis commands. This will always take a few seconds because the db is installing from scratch everytime. If postgres/postgis images aren't already downloaded locally, it will take even longer. Worst case, if the command times out on first run. Stop and remove the parse-postgres container and run the command again, 20 seconds should be enough wait time then
* latest changes
* initial fix, need to test
* fixed lint
* Adding test case for caseInsensitive
Adding test case for verifying indexing for caseInsensitive
* Implementing ensureIndex
* Updated PostgresStorageAdapter calls to ST_DistanceSphere. Note this has a minimum requirement of postgis 2.2. Documented the change in the readme. This is address #6441
* updated postgres sections of contributions with newer postgres info. Also switched postgis image it points to as the other one hasn't been updated in over a year.
* more info about postgres
* added necessary password for postgres docker
* updated wording in contributions
* removed reference to MacJr environment var when starting postgres in contributions. The official image automatically creates a user named 'postgres', but it does require a password, which the command sets to 'postgres'
* added more time to docker sleep/wait to enter postgis commands. This will always take a few seconds because the db is installing from scratch everytime. If postgres/postgis images aren't already downloaded locally, it will take even longer. Worst case, if the command times out on first run. Stop and remove the parse-postgres container and run the command again, 20 seconds should be enough wait time then
* latest changes
* initial fix, need to test
* fixed lint
* Adds caseInsensitive constraints to database, but doesn't pass regular tests. I believe this is because ensureIndex in the Postgres adapter is returning wrong. Also, some issues with the caseInsensitive test case
* this version addes the indexes, but something still wrong with the ensureIndex method in adapter
* removed code from suggestions
* fixed lint
* fixed PostgresAdapter test case
* small bug in test case
* reverted back to main branch package.json and lock file
* fixed docker command in Contribute file
* added ability to explain the find method
* triggering another build
* added ability to choose to 'analyze' a query which actually executes (this can be bad when looking at a query plan for Insert, Delete, etc.) the query or to just setup the query plan (default, previous versions defaulted to 'analyze'). Alse added some comparsons on sequential vs index searches for postgres
* made sure to check that search actually returns 1 result. Removed prep time comparison between searches as this seemed to be variable
* added test cases using find and case insensitivity on fields other than username and password. Also added explain to aggregate method
* fixing issue where query in aggregate replaced the map method incorrectly
* reverted back to mapping for aggregate method to make sure it's the issue
* switched back to caseInsensitive check for email and username as it was causing issues
* fixed aggregate method using explain
* made query plain results more flexible/reusable. Got rid of droptables as 'beforeEach' already handles this
* updated CONTRIBUTING doc to use netrecon as default username for postgres (similar to old style). Note that the official postgres docker image for postgres requires POSTGRES_PASSWORD to be set in order to use the image
* left postgis at 2.5 in the contributing document as this is the last version to be backwards compatibile with older versions of parse server
* updating docker command for postgres
Co-authored-by: Arthur Cinader <700572+acinader@users.noreply.github.com>
* added failing test case
* add date conversion for geoNear query
- geoNear stages were not parsed for date fields, but mongodb nodejs adapter requires date object
* reverted unnecessary code auto-formatting
* limited parsing to query property of geoNear stage
- the geoNear object contains parameter keys which could be identical to field names in the collection, which should not be parsed and changed, therefore restricting parsing only to query parameter key
* reverted unnecessary code auto-formatting
* added index type parameter to ensureIndex
- required to create geo index for geoNear test
* added geo index creation to test case
* fixed dates in test case
- test case likey failed due to date rounding
* added error output to console
- temporary, to find out why test fails on mongodb 3.6.9
* create seperate class to avoid multiple geo indices on TestObject class
- mongodb <4.0 does not allow nultiple geo indices on a class when using geoNear
- see https://docs.mongodb.com/v3.6/reference/operator/aggregation/geoNear/#behavior
* fixed incorrect result validation
- results were not ordered properly, so test validation failed sometimes
* removed error output to console
This reverts commit da81c515cbf8cb6edfd82f09ca3087457ac8c727.
* Attempting to fix Postgres issue
* Attempting to fix Postgres issue
trying to stop loop
* Attempting to fix Postgres
isolating postgres calls
* Attempting to fix Postgres issue
Separating jobs
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
Separating builds again
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
Just added back version 10, just in case it gets called
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Update .travis.yml
* Attempting to fix postgres
Removed postgres installs from unneeded test cases. Added the ability to test Postgres 10 and 11
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
* Attempting to fix postgres
Added test for postgres 12 that's allowed to fail
* Attempting to fix postgres
* Attempting to fix postgres
Second round to see if it fails eventually
* Attempting to fix postgres
Round 3
* Attempting to fix postgres
Allowing all postgres to fail since it seems to occur randomly
* Temporary fix: separated mongo and postgres in travis
Now the mongo and postgres scripts are independent of each other to prevent the `ERROR: could not access file "$libdir/postgis-2.4": No such file or directory` of showing up in the rest of the builds.
In addition, a test for postgres-12 has been added for future compatibility. Both the postgres-11 and postgres-12 have been added to `allow_failures` because the aforementioned error still creeps up. Important note is that the error has nothing to do with compatibility with postgres, but rather seems to be an error of how postgres (or really postgis) is being referenced in the respective travis distribution. Lastly, this error, if truly random should appear less than before as the postgres scripts aren't being run for every build as it previously was running.
* Allowing all postgres to fail
* Allowing multiple names to fail
* Removing preinstalled versions of postgres from list
Seeing if this gets rid of the random error
* Use postgres made for dist
* Second round
* Round 3
* Round 4
* Round 5
* Fixed issue with random postgres fail
Removing the native postgres builds at the right time seems to have fixed the random error from before.
The postgres tests are now not allowed to fail.
* Added back postgres 11 and 12 to allow_failures
The actual problem is fixed, but it seems there are some instability with some of the test cases for postgres that need to be addressed at another time.
The issues that pop up are:
- Postgres-11
```Failures:
1) Cloud Code cloud jobs should set the message / success on the job
Message:
Expected undefined to equal 'hello'.
Stack:
Error: Expected undefined to equal 'hello'.
at <Jasmine>
at req.message.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1571:46)
at process._tickCallback (internal/process/next_tick.js:68:7)
```
- Postgres-12
```
Failures:
1) Cloud Code cloud jobs should set the message / success on the job
Message:
Expected undefined to equal 'hello'.
Stack:
Error: Expected undefined to equal 'hello'.
at <Jasmine>
at req.message.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1571:46)
at process._tickCallback (internal/process/next_tick.js:68:7)
Message:
Expected 'running' to equal 'succeeded'.
Stack:
Error: Expected 'running' to equal 'succeeded'.
at <Jasmine>
at promise.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1580:45)
at process._tickCallback (internal/process/next_tick.js:68:7)
```
* added travis scripts for postgres
* Setting up before_install and before_script
This should shrink the footprint of the file and and reduce the redundancy of calls for postgres.
Added support for testing of Postgres 9 and 10 in the scripts, not adding the tests though
* make scripts executable
* Update .travis.yml
* add sourcing in script
* trying to fix source
* fixing env var in script
* fixed ; near then
* Cleaning up travis file
removed old lines
* Finishing clean up
* Fixing allow_failures since "name" was removed
* Update .travis.yml
* Removed Postgres 11 from allow_failures
* I think using travis default postgres port of 5433 will allow us to not have to remove anything from the image
* Switching travis to postgres port 5433
* modifying script for test
* modifying script for test
* modifying script for test
* reverting back to working way with removing postgres from image
* Reverted back to removing postgres from image
* removing postgres 12
* removed postgres-12 from allow_failures
* updated postgres method from deprecated. Also updating postgis to 3.0
* updated postgis to 3.0
* Update .travis.yml
* Group aggregation supports multiple columns for postgres
* Group aggregation supports multiple columns for postgres
* Group aggregation supports multiple columns for postgres
* Group aggregation supports multiple columns for postgres
* Always delete data after each, even for mongo.
* Add failing simple case test
* run all tests
* 1. when validating username be case insensitive
2. add _auth_data_anonymous to specialQueryKeys...whatever that is!
* More case sensitivity
1. also make email validation case insensitive
2. update comments to reflect what this change does
* wordsmithery and grammar
* first pass at a preformant case insensitive query. mongo only so far.
* change name of parameter from insensitive to
caseInsensitive
* Postgres support
* properly handle auth data null
* wip
* use 'caseInsensitive' instead of 'insensitive' in all places.
* update commenet to reclect current plan
* skip the mystery test for now
* create case insensitive indecies for
mongo to support case insensitive
checks for email and username
* remove unneeded specialKey
* pull collation out to a function.
* not sure what i planned
to do with this test.
removing.
* remove typo
* remove another unused flag
* maintain order
* maintain order of params
* boil the ocean on param sequence
i like having explain last cause it seems
like something you would
change/remove after getting what you want
from the explain?
* add test to verify creation
and use of caseInsensitive index
* add no op func to prostgress
* get collation object from mongocollection
make flow lint happy by declaring things Object.
* fix typo
* add changelog
* kick travis
* properly reference static method
* add a test to confirm that anonymous users with
unique username that do collide when compared
insensitively can still be created.
* minot doc nits
* add a few tests to make sure our spy is working as expected
wordsmith the changelog
Co-authored-by: Diamond Lewis <findlewis@gmail.com>
* added hint to aggregate
* added support for hint in query
* added else clause to aggregate
* fixed tests
* updated tests
* Add tests and clean up
* Add support for explain
Co-authored-by: Diamond Lewis <findlewis@gmail.com>
* Update PostgresStorageAdapter.js
Improving use of the `await.async` notation in relation to `pg-promise`, and in general.
* Update PostgresStorageAdapter.js
* Update PostgresStorageAdapter.js
Correcting some results.
* Update PostgresStorageAdapter.js
* Suppress Test Logs
This will reduce some of the noise in the tests logs.
* replace deprecated buffer
* remove deprecation warnings
* fix geopoint
* Fix GraphQL
* postgres warnings
* added ignore authData field
* add fix for Postgres
* add test for mongoDB
* add test login with provider despite invalid authData
* removed fit
* fixed ignoring authData in postgres
* Fix postgres test
* Throw error instead of ignore
* improve tests
* Add mongo test
* allow authData when not user class
* fix tests
* more tests
* add condition to synthesize authData field only in _User class
it is forbidden to add a custom field name beginning with `_`, so if the object is not `_User` , the transform should throw
* add warning log when ignoring invalid `authData` in `_User`
* add test to throw when custom field begins with underscore
* Fix: aggregate not matching null values
* Exclude Postgres from this new test - it does not even support and is not working correctly - should be addressed separately
