Skip to content

Database: Add autoincrement, uniqueness, and sync-writes polyfills#28

Merged
amotl merged 4 commits into
mainfrom
amo/polyfill
Jun 24, 2024
Merged

Database: Add autoincrement, uniqueness, and sync-writes polyfills#28
amotl merged 4 commits into
mainfrom
amo/polyfill

Conversation

@amotl

@amotl amotl commented Jan 16, 2024

Copy link
Copy Markdown
Contributor

About

Add a few polyfills conceived on behalf of the LangChain adapter, and patches to rdflib-sqlalchemy as well as meltano-target-cratedb. By adding them here, they can be used by downstream applications without needing to install cratedb-toolkit.

Documentation

Preview: https://sqlalchemy-cratedb--28.org.readthedocs.build/support.html

@simonprickett: I would also be happy about your review on documentation syntax/grammar/wording, as I am not an English native speaker. Thanks!

References

Backlog

  • Software tests.
  • Documentation.

Comment thread src/sqlalchemy_cratedb/polyfill.py Outdated
@amotl amotl force-pushed the amo/polyfill branch 4 times, most recently from 9cc237c to 8158fa6 Compare June 18, 2024 23:01
Comment thread src/sqlalchemy_cratedb/support/polyfill.py Fixed
Comment thread src/sqlalchemy_cratedb/support/polyfill.py Fixed
Comment thread src/sqlalchemy_cratedb/support/polyfill.py Fixed
@amotl amotl force-pushed the amo/polyfill branch 4 times, most recently from 6b65a19 to 600f8ac Compare June 19, 2024 01:33
Comment thread src/sqlalchemy_cratedb/support/util.py Fixed

@matriv matriv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx @amotl !
Nice stuff, left some suggestions on the documentation.

Comment thread CHANGES.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated
@amotl amotl requested a review from seut June 20, 2024 19:48

@matriv matriv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx! LGTM, but please wait for one more review.

Comment thread docs/support.md Outdated
Comment thread docs/index.rst Outdated

The package bundles a few support and utility functions that try to fill a few
gaps you will observe when working with CrateDB, when compared with other
databases. As a distributed OLAP database, it naturally does not include certain

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we shouldn't mention as a distributed OLAP, specially combining it with naturally after that, I would simplify and say that CrateDB's behavior and features deviate from those found usually in an RDBMS.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like "Due to its distributed nature, CrateDB's behavior and features differ from those found in other RDBMS systems".

Comment thread docs/overview.rst Outdated
Comment thread docs/overview.rst Outdated
which enable automatically assigning incremental values when inserting records.
However, it offers server-side support by providing an SQL function to generate
random identifiers of ``STRING`` type, and client-side support for generating
``INTEGER``-based identifiers on behalf of the SQLAlchemy dialect.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what "on behalf of" means here - does this mean "when using the SQLAlchemy dialect" or something else? "on behalf of" implies there's some ID generating function that can be plugged into SQLAlchemy. I don't know SQLAlchemy so am not sure what happens here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"when using the SQLAlchemy dialect" is better. Thanks.

Comment thread docs/support.md Outdated
Comment thread docs/support.md Outdated

@simonprickett simonprickett left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple of small suggestions.

Comment thread docs/support.md Outdated
amotl and others added 4 commits June 24, 2024 16:13
Sources: MLflow, LangChain, Singer/Meltano, rdflib-sqlalchemy
Co-authored-by: Marios Trivyzas <5058131+matriv@users.noreply.github.com>
Co-authored-by: Simon Prickett <simon@crudworks.org>
@amotl

amotl commented Jun 24, 2024

Copy link
Copy Markdown
Contributor Author

@matriv and @simonprickett: Thanks a stack for your excellent reviews. 💯

@amotl amotl merged commit 1549fe6 into main Jun 24, 2024
@amotl amotl deleted the amo/polyfill branch June 24, 2024 14:27
Comment on lines +111 to +113
full_table_name = f'"{clauseelement.table.name}"'
if clauseelement.table.schema is not None:
full_table_name = f'"{clauseelement.table.schema}".' + full_table_name

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer moving this quoting logic into the refresh_tablefunction, by change it's signature to consume an optional schema. This would avoid that it will be called with unquoted or invalid relation names.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should work. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants