.. Automatically generated by code2rst.py
   Edit src/session.c not this file!

.. currentmodule:: apsw

Session extension
*****************

APSW provides access to all session functionality (including
experimental).  See the :doc:`example-session`.

The `session extension <https://www.sqlite.org/sessionintro.html>`__
allows recording changes to a database, and later replaying them on
another database, or undoing them.  This allows offline syncing, as
well as collaboration.  It is also useful for debugging, development,
and testing.  Note that it records the added, modified, and deleted
row values - it does **not** record or replay the queries that
resulted in those changes.

* You can choose which tables have changes recorded (or all), and
  pause / resume recording at any time

* The recorded change set includes the row values before and after a
  change.  This allows comprehensive conflict detection, and inverting
  (undoing the change),  Optionally you can use patch sets (a subset of
  change sets) which do not have the before values, consuming less
  space but have less ability to detect conflicts, or be inverted.

* The recorded changes includes indirect changes made such as by triggers
  and foreign keys.

* When applying changes you can supply a conflict handler to choose
  what happens on each conflicting row, including aborting, skipping,
  applying anyway, applying your own change, and can record the
  conflicting operation to another change set for later.

* You are responsible for :ref:`managing your schema <schema_upgrade>`
  - the extension will not create, update, or delete tables for you.
  When applying changesets, if a corresponding table does not already
  exist then those changes are ignored.  This means that you do not
  need all tables present on all databases.

* It is efficient only storing enough to make the semantic change.
  For example if multiple changes are made to the same row, then
  they can be accumulated into one change record, not many.

* You can iterate over a change set to see what it contains

* Changesets do not contain the changes in the order made

* Using :class:`ChangesetBuilder`, you can accumulate multiple change
  sets, and add changes from an iterator or conflict handler.

* Using :class:`Rebaser` you can merge conflict resolutions made when
  applying a changeset into a later changeset, so those conflict
  resolutions do not have to be redone on each database where they are
  applied.

* Doing multi-way synchronization across multiple databases changed
  separately `is hard
  <https://en.wikipedia.org/wiki/Eventual_consistency>`__.  A common
  approach to conflicts is to use timestamps with the most recent
  change "winning".  Changesets do not include timestamps, and are not
  time ordered.  You should carefully design your schema and
  synchronization to ensure the needed levels of data integrity,
  consistency, and meeting user goals up front.  Adding it later is
  painful.

* Most APIs produce and consume changesets as bytes (or :class:`bytes
  like <collections.abc.Buffer>`). That limits the changeset size to
  2GB - the limit is in the SQLite code and also the limit for `blobs
  <https://www.sqlite.org/limits.html>`__.  To produce or consume
  larger changesets, or to not have an entire changeset in memory,
  there are streaming versions of most APIs where you need to provide
  to provide a :class:`block input <SessionStreamInput>` or
  :class:`block output <SessionStreamOutput>` callback.

.. important::

    By default Session can only record and replay changes that have an
    explicit `primary key <https://www.sqlite.org/lang_createtable.html#the_primary_key>`__
    defined (ie ``PRIMARY KEY`` must be present in the table definition).
    It doesn't matter what type or how many columns make up the primary key.
    This provides a stable way to identify rows for insertion, changes, and
    deletion.

    You can use :meth:`Session.config` with `SQLITE_SESSION_OBJCONFIG_ROWID
    <https://www.sqlite.org/session/c_session_objconfig_rowid.html>`__
    to enable recording of tables without an explicit primary key, but
    it is strongly advised to have deterministic primary keys so that
    changes made independently can be reconciled.  The changesets will
    also contain wrong operations if the table has a column named
    `_rowid_`.

Availability
============

The session extension and APSW support for it have to be enabled at
compile time for each.  APSW builds from PyPI include session support.

Most platform provided SQLite are configured with session support, and
APSW should end up with it too.

The methods and classes documented here are only present if session
support was enabled.

Usage Overview
==============

The session extension does not do table creation (or deletion).  When applying
a changeset, it will only do so if a same named table exists, with the same number
of columns, and same primary key.  If no such table exists, the change is silently
ignored.  (Tip for :ref:`managing your schema <schema_upgrade>`)

To record changes:

* Use a :class:`Session` with the relevant database.  You can
  have multiple on the same database.
* Use :meth:`Session.attach` to determine which tables
  to record
* You can use :attr:`Session.enabled` to turn recording off or
  on (it is on by default)
* Use :meth:`Session.changeset` to get the changeset for later use.
* If you have two databases, you can use :meth:`Session.diff` to get
  the changes necessary to turn one into the other without having to
  record changes as they happen

To see what your changeset contains:

* Use :meth:`Changeset.iter`

To apply a changeset:

* Use :meth:`Changeset.apply`

To manipulate changesets:

* Use :class:`ChangesetBuilder`
* You can add multiple changesets together
* You can add :class:`individual changes <TableChange>` from
  :meth:`Changeset.iter` or from your conflict handler in
  :meth:`Changeset.apply`
* Use :class:`Rebaser` to incorporate conflict resolutions into a
  changeset

.. tip::

  The session extension rarely raises exceptions, instead just doing
  nothing.  For example if tables don't exist, don't have a primary key,
  attached databases don't exist, and similar scenarios where typos
  could happen, you won't get an error, just no action.

Extension configuration
=======================

.. index:: sqlite3session_config

.. method:: session_config(op: int, *args: Any) -> Any

 :param op: One of the `sqlite3session options <https://www.sqlite.org/session/c_session_config_strmsize.html>`__
 :param args: Zero or more arguments as appropriate for *op*

 Calls: `sqlite3session_config <https://sqlite.org/session/sqlite3session_config.html>`__

Session class
=============

.. index:: sqlite3session_create

.. class:: Session(db: Connection, schema: str)

  This object wraps a `sqlite3_session
  <https://www.sqlite.org/session/session.html>`__ object.

  Starts a new session.

  :param connection: Which database to operate on
  :param schema: `main`, `temp`, the name in `ATTACH <https://sqlite.org/lang_attach.html>`__

  Calls: `sqlite3session_create <https://sqlite.org/session/sqlite3session_create.html>`__

.. index:: sqlite3session_attach

.. method:: Session.attach(name: Optional[str] = None) -> None

 Attach to a specific table, or all tables if no name is provided.  The
 table does not need to exist at the time of the call.  You can call
 this multiple times.

 .. seealso::

    :meth:`table_filter`

 Calls: `sqlite3session_attach <https://sqlite.org/session/sqlite3session_attach.html>`__

.. index:: sqlite3session_changeset

.. method:: Session.changeset() -> bytes

  Produces a changeset of the session so far.

  Calls: `sqlite3session_changeset <https://sqlite.org/session/sqlite3session_changeset.html>`__

.. index:: sqlite3session_changeset_size

.. attribute:: Session.changeset_size
    :type: int

    Returns upper limit on changeset size, but only if :meth:`Session.config`
    was used to enable it.  Otherwise it will be zero.

    Calls: `sqlite3session_changeset_size <https://sqlite.org/session/sqlite3session_changeset_size.html>`__

.. index:: sqlite3session_changeset_strm

.. method:: Session.changeset_stream(output: SessionStreamOutput) -> None

  Produces a changeset of the session so far in a stream

  Calls: `sqlite3session_changeset_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3session_delete

.. method:: Session.close() -> None

  Ends the session object.  APSW ensures that all
  Session objects are closed before the database is closed
  so there is no need to manually call this.

  Calls: `sqlite3session_delete <https://sqlite.org/session/sqlite3session_delete.html>`__

.. index:: sqlite3session_object_config

.. method:: Session.config(op: int, *args: Any) -> Any

  Set or get `configuration values <https://www.sqlite.org/session/c_session_objconfig_rowid.html>`__

  For example :code:`session.config(apsw.SQLITE_SESSION_OBJCONFIG_SIZE, -1)` tells you
  if size information is enabled.

  Calls: `sqlite3session_object_config <https://sqlite.org/session/sqlite3session_object_config.html>`__

.. index:: sqlite3session_diff

.. method:: Session.diff(from_schema: str, table: str) -> None

  Loads the changes necessary to update the named ``table`` in the attached database
  ``from_schema`` to match the same named table in the database this session is
  attached to.

  See the :ref:`example <example_session_diff>`.

  .. note::

    You must use :meth:`attach` (or use :meth:`table_filter`) to attach to
    the table before running this method otherwise nothing is recorded.

  Calls: `sqlite3session_diff <https://sqlite.org/session/sqlite3session_diff.html>`__

.. index:: sqlite3session_enable

.. attribute:: Session.enabled
    :type: bool

    Get or change if this session is recording changes.  Disabling only
    stops recording rows not already part of the changeset.

    Calls: `sqlite3session_enable <https://sqlite.org/session/sqlite3session_enable.html>`__

.. index:: sqlite3session_indirect

.. attribute:: Session.indirect
    :type: bool

    Get or change if this session is in indirect mode

    Calls: `sqlite3session_indirect <https://sqlite.org/session/sqlite3session_indirect.html>`__

.. index:: sqlite3session_isempty

.. attribute:: Session.is_empty
    :type: bool

    True if no changes have been recorded.

    Calls: `sqlite3session_isempty <https://sqlite.org/session/sqlite3session_isempty.html>`__

.. index:: sqlite3session_memory_used

.. attribute:: Session.memory_used
    :type: int

    How many bytes of memory have been used to record session changes.

    Calls: `sqlite3session_memory_used <https://sqlite.org/session/sqlite3session_memory_used.html>`__

.. index:: sqlite3session_patchset

.. method:: Session.patchset() -> bytes

  Produces a patchset of the session so far.  Patchsets do not include
  before values of changes, making them smaller, but also harder to detect
  conflicts.

  Calls: `sqlite3session_patchset <https://sqlite.org/session/sqlite3session_patchset.html>`__

.. index:: sqlite3session_patchset_strm

.. method:: Session.patchset_stream(output: SessionStreamOutput) -> None

  Produces a patchset of the session so far in a stream

  Calls: `sqlite3session_patchset_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3session_table_filter

.. method:: Session.table_filter(callback: Callable[[str], bool]) -> None

  Register a callback that says if changes to the named table should be
  recorded.  If your callback has an exception then ``False`` is
  returned.

  .. seealso::

    :meth:`attach`

  Calls: `sqlite3session_table_filter <https://sqlite.org/session/sqlite3session_table_filter.html>`__

TableChange class
=================

.. class:: TableChange

  Represents a `changed row
  <https://sqlite.org/session/changeset_iter.html>`__.  They come from
  :meth:`changeset iteration <Changeset.iter>` and from the
  :meth:`conflict handler in apply <Changeset.apply>`.

  A TableChange is only valid when your conflict handler is active, or
  has just been provided by a changeset iterator.  It goes out of scope
  after your conflict handler returns, or the iterator moves to the next
  entry.  You will get :exc:`~apsw.InvalidContextError` if you try to
  access fields when out of scope.  This means you can't save
  TableChanges for later, and need to copy out any information you need.

.. attribute:: TableChange.column_count
  :type: int

   Number of columns in the affected table

.. index:: sqlite3changeset_conflict

.. attribute:: TableChange.conflict
  :type: tuple[SQLiteValue, ...] | None

  :class:`None` if not applicable (not in a conflict).  Otherwise a
  tuple of values for the conflicting row.

  Calls: `sqlite3changeset_conflict <https://sqlite.org/session/sqlite3changeset_conflict.html>`__

.. index:: sqlite3changeset_fk_conflicts

.. attribute:: TableChange.fk_conflicts
  :type: int | None

  The number of known foreign key conflicts, or :class:`None` if not in a
  conflict handler.

  Calls: `sqlite3changeset_fk_conflicts <https://sqlite.org/session/sqlite3changeset_fk_conflicts.html>`__

.. attribute:: TableChange.indirect
  :type: bool

  ``True`` if this is an `indirect <https://sqlite.org/session/sqlite3session_indirect.html>`__
  change - for example made by triggers or foreign keys.

.. attribute:: TableChange.name
  :type: str

   Name of the affected table

.. index:: sqlite3changeset_new

.. attribute:: TableChange.new
  :type: tuple[SQLiteValue | Literal[no_change], ...] | None

  :class:`None` if not applicable (like a DELETE).  Otherwise a
  tuple of the new values for the row, with :attr:`apsw.no_change`
  if no value was provided for that column.

  Calls: `sqlite3changeset_new <https://sqlite.org/session/sqlite3changeset_new.html>`__

.. index:: sqlite3changeset_old

.. attribute:: TableChange.old
  :type: tuple[SQLiteValue | Literal[no_change], ...] | None

  :class:`None` if not applicable (like an INSERT).  Otherwise a tuple
  of the old values for the row before this change, with
  :attr:`apsw.no_change` if no value was provided for that column,

  Calls: `sqlite3changeset_old <https://sqlite.org/session/sqlite3changeset_old.html>`__

.. attribute:: TableChange.op
  :type: str

   The operation code as a string  ``INSERT``,
   ``DELETE``, or ``UPDATE``.  See :attr:`opcode`
   for this as a number.

.. attribute:: TableChange.opcode
  :type: int

   The operation code - ``apsw.SQLITE_INSERT``,
   ``apsw.SQLITE_DELETE``, or ``apsw.SQLITE_UPDATE``.
   See :attr:`op` for this as a string.

.. index:: sqlite3changeset_pk

.. attribute:: TableChange.pk_columns
  :type: set[int]

  Which columns make up the primary key for this table

  Calls: `sqlite3changeset_pk <https://sqlite.org/session/sqlite3changeset_pk.html>`__

Changeset class
===============

.. class:: Changeset

  Provides changeset (including patchset) related methods.  Note that
  all methods are static (belong to the class).  There is no Changeset
  object.   On input Changesets can be a :class:`collections.abc.Buffer`
  (anything that resembles a sequence of bytes), or
  :class:`SessionStreamInput` which provides the bytes in chunks from a
  callback.

  Output is bytes, or :class:`SessionStreamOutput` (chunks in a callback).

  The streaming versions are useful when you are concerned about memory
  usage, or where changesets are larger than 2GB (the SQLite limit).

.. index:: sqlite3changeset_apply_v2, sqlite3changeset_apply_v2_strm

.. method:: Changeset.apply(changeset: ChangesetInput, db: Connection, *, filter: Optional[Callable[[str], bool]] = None, conflict: Optional[Callable[[int,TableChange], int]] = None, flags: int = 0, rebase: bool = False) -> bytes | None

  Applies a changeset to a database.

  :param source: The changeset either as the bytes, or a stream
  :param db: The connection to make the change on
  :param filter: Callback to determine if changes to a table are done
  :param conflict: Callback to handle a change that cannot be applied
  :param flags: `v2 API flags <https://www.sqlite.org/session/c_changesetapply_fknoaction.html>`__.
  :param rebase: If ``True`` then return :class:`rebase <Rebaser>` information, else :class:`None`.

  Filter
  ------

  Callback called with a table name, once per table that has a change.  It should return ``True``
  if changes to that table should be applied, or ``False`` to ignore them.  If not supplied then
  all tables have changes applied.

  Conflict
  --------

  When a change cannot be applied the conflict handler determines what
  to do.  It is called with a `conflict reason
  <https://www.sqlite.org/session/c_changeset_conflict.html>`__ as the
  first parameter, and a :class:`TableChange` as the second.  Possible
  conflicts are `described here
  <https://sqlite.org/sessionintro.html#conflicts>`__.

  It should return the `action to take <https://www.sqlite.org/session/c_changeset_abort.html>`__.

  If not supplied or on error, ``SQLITE_CHANGESET_ABORT`` is returned.

  See the :ref:`example <example_applying>`.

  Calls:
    * `sqlite3changeset_apply_v2 <https://sqlite.org/session/sqlite3changeset_apply.html>`__
    * `sqlite3changeset_apply_v2_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3changeset_concat

.. method:: Changeset.concat(A: collections.abc.Buffer, B: collections.abc.Buffer) -> bytes

  Returns combined changesets

  Calls: `sqlite3changeset_concat <https://sqlite.org/session/sqlite3changeset_concat.html>`__

.. index:: sqlite3changeset_concat_strm

.. method:: Changeset.concat_stream(A: SessionStreamInput, B: SessionStreamInput, output: SessionStreamOutput) -> None

  Streaming concatenate two changesets

  Calls: `sqlite3changeset_concat_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3changeset_invert

.. method:: Changeset.invert(changeset: collections.abc.Buffer) -> bytes

  Produces a changeset that reverses the effect of
  the supplied changeset.

  Calls: `sqlite3changeset_invert <https://sqlite.org/session/sqlite3changeset_invert.html>`__

.. index:: sqlite3changeset_invert_strm

.. method:: Changeset.invert_stream(changeset: SessionStreamInput, output: SessionStreamOutput) -> None

  Streaming reverses the effect of the supplied changeset.

  Calls: `sqlite3changeset_invert_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3changeset_start, sqlite3changeset_start_v2, sqlite3changeset_start_strm, sqlite3changeset_start_v2_strm

.. method:: Changeset.iter(changeset: ChangesetInput, *, flags: int = 0) -> Iterator[TableChange]

   Provides an iterator over a changeset.  You can supply the changeset as
   the bytes, or streamed via a callable.

   If flags is non-zero them the ``v2`` API is used (marked as experimental)

   Calls:
     * `sqlite3changeset_start <https://sqlite.org/session/sqlite3changeset_start.html>`__
     * `sqlite3changeset_start_v2 <https://sqlite.org/session/sqlite3changeset_start.html>`__
     * `sqlite3changeset_start_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__
     * `sqlite3changeset_start_v2_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

ChangesetBuilder class
======================

.. index:: sqlite3changegroup_new

.. class:: ChangesetBuilder()

  This object wraps a `sqlite3_changegroup <https://sqlite.org/session/changegroup.html>`__
  letting you concatenate changesets and individual :class:`TableChange` into one larger
  changeset.

 Creates a new empty builder.

 Calls: `sqlite3changegroup_new <https://sqlite.org/session/sqlite3changegroup_new.html>`__

.. index:: sqlite3changegroup_add, sqlite3changegroup_add_strm

.. method:: ChangesetBuilder.add(changeset: ChangesetInput) -> None

  :param changeset: The changeset as the bytes, or a stream

  Adds the changeset to the builder

  Calls:
    * `sqlite3changegroup_add <https://sqlite.org/session/sqlite3changegroup_add.html>`__
    * `sqlite3changegroup_add_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3changegroup_add_change

.. method:: ChangesetBuilder.add_change(change: TableChange) -> None

  :param change: An individual change to add.

  You can obtain :class:`TableChange` from :meth:`Changeset.iter` or from the conflict callback
  of :meth:`Changeset.apply`.

  Calls: `sqlite3changegroup_add_change <https://sqlite.org/session/sqlite3changegroup_add_change.html>`__

.. index:: sqlite3changegroup_delete

.. method:: ChangesetBuilder.close() -> None

  Releases the builder

  Calls: `sqlite3changegroup_delete <https://sqlite.org/session/sqlite3changegroup_delete.html>`__

.. index:: sqlite3changegroup_output

.. method:: ChangesetBuilder.output() -> bytes

  Produces a changeset of what was built so far

  Calls: `sqlite3changegroup_output <https://sqlite.org/session/sqlite3changegroup_output.html>`__

.. index:: sqlite3changegroup_output_strm

.. method:: ChangesetBuilder.output_stream(output: SessionStreamOutput) -> None

  Produces a streaming changeset of what was built so far

  Calls: `sqlite3changegroup_output_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__

.. index:: sqlite3changegroup_schema

.. method:: ChangesetBuilder.schema(db: Connection, schema: str) -> None

  Ensures the changesets comply with the tables in the database

  :param db: Connection to consult
  :param schema: `main`, `temp`, the name in `ATTACH <https://sqlite.org/lang_attach.html>`__

  You will get :exc:`MisuseError` if changes have already been added, or this method has
  already been called.

  Calls: `sqlite3changegroup_schema <https://sqlite.org/session/sqlite3changegroup_schema.html>`__

Rebaser class
=============

.. index:: sqlite3rebaser_create

.. class:: Rebaser()

  This object wraps a `sqlite3_rebaser
  <https://www.sqlite.org/session/rebaser.html>`__ object.

  Starts a new rebaser.

  Calls: `sqlite3rebaser_create <https://sqlite.org/session/sqlite3rebaser_create.html>`__

.. index:: sqlite3rebaser_configure

.. method:: Rebaser.configure(cr: collections.abc.Buffer) -> None

  Tells the rebaser about conflict resolutions made in an earlier
  :meth:`Changeset.apply`.

  Calls: `sqlite3rebaser_configure <https://sqlite.org/session/sqlite3rebaser_configure.html>`__

.. index:: sqlite3rebaser_rebase

.. method:: Rebaser.rebase(changeset: collections.abc.Buffer) -> bytes

  Produces a new changeset rebased according to :meth:`configure` calls made.

  Calls: `sqlite3rebaser_rebase <https://sqlite.org/session/sqlite3rebaser_rebase.html>`__

.. index:: sqlite3rebaser_rebase_strm

.. method:: Rebaser.rebase_stream(changeset: SessionStreamInput, output: SessionStreamOutput) -> None

  Produces a new changeset rebased according to :meth:`configure` calls made, using streaming
  input and output.

  Calls: `sqlite3rebaser_rebase_strm <https://sqlite.org/session/sqlite3changegroup_add_strm.html>`__