.. Automatically generated by code2rst.py Edit src/vfs.c not this file! .. currentmodule:: apsw .. _vfs: Virtual File System (VFS) ************************* `VFS `__ defines the interface between the SQLite core and the underlying operating system. The majority of the functionality deals with files. APSW exposes this functionality letting you provide your own routines. You can also *inherit* from an existing vfs making it easy to augment or override specific routines. You specify which VFS to use as a parameter to the :class:`Connection` constructor. .. code-block:: python db=apsw.Connection("file", vfs="myvfs") The easiest way to get started is to make a :class:`VFS` derived class that inherits from the default vfs. Then override methods you want to change behaviour of. If you want to just change how file operations are done then you have to override :meth:`VFS.xOpen` to return a file instance that has your overridden :class:`VFSFile` methods. The :ref:`example ` demonstrates obfuscating the database file contents. Exceptions and errors ===================== To return an error from any routine you should raise an exception. The exception will be translated into the corresponding SQLite error code. To return a specific SQLite error code use :meth:`exception_for`. If the exception does not map to any specific error code then *SQLITE_ERROR* which corresponds to :exc:`SQLError` is returned to SQLite. The SQLite code that deals with VFS errors behaves in varying ways. Some routines have no way to return an error: eg `xDlOpen `_ just returns zero/NULL on being unable to load a library, `xSleep `_ has no error return parameter), others are unified (eg almost any error in xWrite will be returned to the user as disk full error). Sometimes errors are ignored as they are harmless such as when a journal can't be deleted after a commit (the journal is marked as obsolete before being deleted). Simple operations such as opening a database can result in many different VFS function calls such as hot journals being detected, locking, and read/writes for playback/rollback. If multiple exceptions occur during the same SQLite control flow, then they will be :doc:`chained ` together. :ref:`Augmented stack traces ` are available which significantly increase detail about the exceptions and help with debugging. VFSFcntlPragma class ==================== .. class:: VFSFcntlPragma(pointer: int) A helper class to work with `SQLITE_FCNTL_PRAGMA `__ in :meth:`VFSFile.xFileControl`. The :ref:`example ` shows usage of this class. It is only valid while in :meth:`VFSFile.xFileControl`, and using outside of that will result in memory corruption and crashes. The pointer must be what your xFileControl method received. .. attribute:: VFSFcntlPragma.name :type: str The name of the pragma .. attribute:: VFSFcntlPragma.result :type: str | None The first element which becomes the result or error message .. attribute:: VFSFcntlPragma.value :type: str | None The value for the pragma, if provided else None, VFS class ========= .. index:: sqlite3_vfs_register, sqlite3_vfs_find .. class:: VFS(name: str, base: Optional[str] = None, makedefault: bool = False, maxpathname: int = 1024, *, iVersion: int = 3, exclude: Optional[set[str]] = None) Provides operating system access. You can get an overview in the `SQLite documentation `_. To create a VFS your Python class must inherit from :class:`VFS`. :param name: The name to register this vfs under. If the name already exists then this vfs will replace the prior one of the same name. Use :meth:`apsw.vfs_names` to get a list of registered vfs names. :param base: If you would like to inherit behaviour from an already registered vfs then give their name. To inherit from the default vfs, use a zero length string ``""`` as the name. :param makedefault: If true then this vfs will be registered as the default, and will be used by any opens that don't specify a vfs. :param maxpathname: The maximum length of database name in bytes when represented in UTF-8. If a pathname is passed in longer than this value then SQLite will not`be able to open it. If you are using a base, then a value of zero will use the value from base. :param iVersion: Version number for the `sqlite3_vfs `__ structure. :param exclude: A set of strings, naming the methods that will be filled in with ``NULL`` in the `sqlite3_vfs `__ structure to indicate to SQLite that they are not supported. Calls: * `sqlite3_vfs_register `__ * `sqlite3_vfs_find `__ .. method:: VFS.excepthook(etype: type[BaseException], evalue: BaseException, etraceback: Optional[types.TracebackType]) -> Any Called when there has been an exception in a :class:`VFS` routine, and it can't be reported to the caller as usual. The default implementation passes the exception information to sqlite3_log, and the first non-error of :func:`sys.unraisablehook` and :func:`sys.excepthook`, falling back to `PyErr_Display`. .. index:: sqlite3_vfs_unregister .. method:: VFS.unregister() -> None Unregisters the VFS making it unavailable to future database opens. You do not need to call this as the VFS is automatically unregistered by when the VFS has no more references or open databases using it. It is however useful to call if you have made your VFS be the default and wish to immediately make it be unavailable. It is safe to call this routine multiple times. Calls: `sqlite3_vfs_unregister `__ .. method:: VFS.xAccess(pathname: str, flags: int) -> bool SQLite wants to check access permissions. Return True or False accordingly. :param pathname: File or directory to check :param flags: One of the `access flags `_ .. method:: VFS.xCurrentTime() -> float Return the `Julian Day Number `_ as a floating point number where the integer portion is the day and the fractional part is the time. .. method:: VFS.xCurrentTimeInt64() -> int Returns as an integer the `Julian Day Number `__ multiplied by 86400000 (the number of milliseconds in a 24-hour day). .. method:: VFS.xDelete(filename: str, syncdir: bool) -> None Delete the named file. If the file is missing then raise an :exc:`IOError` exception with extendedresult *SQLITE_IOERR_DELETE_NOENT* :param filename: File to delete :param syncdir: If True then the directory should be synced ensuring that the file deletion has been recorded on the disk platters. ie if there was an immediate power failure after this call returns, on a reboot the file would still be deleted. .. method:: VFS.xDlClose(handle: int) -> None Close and unload the library corresponding to the handle you returned from :meth:`~VFS.xDlOpen`. You can use ctypes to do this:: def xDlClose(handle: int): # Note leading underscore in _ctypes _ctypes.dlclose(handle) # Linux/Mac/Unix _ctypes.FreeLibrary(handle) # Windows .. method:: VFS.xDlError() -> str Return an error string describing the last error of :meth:`~VFS.xDlOpen` or :meth:`~VFS.xDlSym` (ie they returned zero/NULL). If you do not supply this routine then SQLite provides a generic message. To implement this method, catch exceptions in :meth:`~VFS.xDlOpen` or :meth:`~VFS.xDlSym`, turn them into strings, save them, and return them in this routine. If you have an error in this routine or return None then SQLite's generic message will be used. .. method:: VFS.xDlOpen(filename: str) -> int Load the shared library. You should return a number which will be treated as a void pointer at the C level. On error you should return 0 (NULL). The number is passed as is to :meth:`~VFS.xDlSym`/:meth:`~VFS.xDlClose` so it can represent anything that is convenient for you (eg an index into an array). You can use ctypes to load a library:: def xDlOpen(name: str): return ctypes.cdll.LoadLibrary(name)._handle .. method:: VFS.xDlSym(handle: int, symbol: str) -> int Returns the address of the named symbol which will be called by SQLite. On error you should return 0 (NULL). You can use ctypes:: def xDlSym(ptr: int, name: str): return _ctypes.dlsym (ptr, name) # Linux/Unix/Mac etc (note leading underscore) return ctypes.win32.kernel32.GetProcAddress (ptr, name) # Windows :param handle: The value returned from an earlier :meth:`~VFS.xDlOpen` call :param symbol: A string .. method:: VFS.xFullPathname(name: str) -> str Return the absolute pathname for name. You can use ``os.path.abspath`` to do this. .. method:: VFS.xGetLastError() -> tuple[int, str] Return an integer error code and (optional) text describing the last error code and message that happened in this thread. .. method:: VFS.xGetSystemCall(name: str) -> Optional[int] Returns a pointer for the current method implementing the named system call. Return None if the call does not exist. .. method:: VFS.xNextSystemCall(name: Optional[str]) -> Optional[str] This method is repeatedly called to iterate over all of the system calls in the vfs. When called with None you should return the name of the first system call. In subsequent calls return the name after the one passed in. If name is the last system call then return None. .. method:: VFS.xOpen(name: Optional[str | URIFilename], flags: list[int,int]) -> VFSFile This method should return a new file object based on name. You can return a :class:`VFSFile` from a completely different VFS. :param name: File to open. Note that *name* may be *None* in which case you should open a temporary file with a name of your choosing. May be an instance of :class:`URIFilename`. :param flags: A list of two integers ``[inputflags, outputflags]``. Each integer is one or more of the `open flags `_ binary orred together. The ``inputflags`` tells you what SQLite wants. For example *SQLITE_OPEN_DELETEONCLOSE* means the file should be automatically deleted when closed. The ``outputflags`` describes how you actually did open the file. For example if you opened it read only then *SQLITE_OPEN_READONLY* should be set. .. method:: VFS.xRandomness(numbytes: int) -> bytes This method is called once on the default VFS when SQLite needs to seed the random number generator. You can return less than the number of bytes requested including None. If you return more then the surplus is ignored. .. method:: VFS.xSetSystemCall(name: Optional[str], pointer: int) -> bool Change a system call used by the VFS. This is useful for testing and some other scenarios such as sandboxing. :param name: The string name of the system call :param pointer: A pointer provided as an int. There is no reference counting or other memory tracking of the pointer. If you provide one you need to ensure it is around for the lifetime of this and any other related VFS. Raise an exception to return an error. If the system call does not exist then raise :exc:`NotFoundError`. If `name` is None, then all systemcalls are reset to their defaults. :returns: True if the system call was set. False if the system call is not known. .. method:: VFS.xSleep(microseconds: int) -> int Pause execution of the thread for at least the specified number of microseconds (millionths of a second). This routine is typically called from the busy handler. :returns: How many microseconds you actually requested the operating system to sleep for. For example if your operating system sleep call only takes seconds then you would have to have rounded the microseconds number up to the nearest second and should return that rounded up value. VFSFile class ============= .. class:: VFSFile(vfs: str, filename: str | URIFilename | None, flags: list[int,int]) Wraps access to a file. You only need to derive from this class if you want the file object returned from :meth:`VFS.xOpen` to inherit from an existing VFS implementation. :param vfs: The vfs you want to inherit behaviour from. You can use an empty string ``""`` to inherit from the default vfs. :param name: The name of the file being opened. May be an instance of :class:`URIFilename`. :param flags: A two item list ``[inflags, outflags]`` as detailed in :meth:`VFS.xOpen`. :raises ValueError: If the named VFS is not registered. .. note:: If the VFS that you inherit from supports :ref:`write ahead logging ` then your :class:`VFSFile` will also support the xShm methods necessary to implement wal. .. seealso:: :meth:`VFS.xOpen` .. method:: VFSFile.excepthook(etype: type[BaseException], evalue: BaseException, etraceback: Optional[types.TracebackType]) ->None Called when there has been an exception in a :class:`VFSFile` routine, and it can't be reported to the caller as usual. The default implementation passes the exception information to sqlite3_log, and the first non-error of :func:`sys.unraisablehook` and :func:`sys.excepthook`, falling back to `PyErr_Display`. .. method:: VFSFile.xCheckReservedLock() -> bool Returns True if any database connection (in this or another process) has a lock other than `SQLITE_LOCK_NONE or SQLITE_LOCK_SHARED `_. .. method:: VFSFile.xClose() -> None Close the database. Note that even if you return an error you should still close the file. It is safe to call this method multiple times. .. method:: VFSFile.xDeviceCharacteristics() -> int Return `I/O capabilities `_ (bitwise or of appropriate values). If you do not implement the function or have an error then 0 (the SQLite default) is returned. .. method:: VFSFile.xFileControl(op: int, ptr: int) -> bool Receives `file control `_ request typically issued by :meth:`Connection.file_control`. See :meth:`Connection.file_control` for an example of how to pass a Python object to this routine. :param op: A numeric code. Codes below 100 are reserved for SQLite internal use. :param ptr: An integer corresponding to a pointer at the C level. :returns: A boolean indicating if the op was understood Ensure you pass any unrecognised codes through to your super class. For example:: def xFileControl(self, op: int, ptr: int) -> bool: if op == 1027: process_quick(ptr) elif op == 1028: obj=ctypes.py_object.from_address(ptr).value else: # this ensures superclass implementation is called return super().xFileControl(op, ptr) # we understood the op return True .. method:: VFSFile.xFileSize() -> int Return the size of the file in bytes. .. method:: VFSFile.xLock(level: int) -> None Increase the lock to the level specified which is one of the `SQLITE_LOCK `_ family of constants. If you can't increase the lock level because someone else has locked it, then raise :exc:`BusyError`. .. method:: VFSFile.xRead(amount: int, offset: int) -> bytes Read the specified *amount* of data starting at *offset*. You should make every effort to read all the data requested, or return an error. If you have the file open for non-blocking I/O or if signals happen then it is possible for the underlying operating system to do a partial read. You will need to request the remaining data. Except for empty files SQLite considers short reads to be a fatal error. :param amount: Number of bytes to read :param offset: Where to start reading. .. method:: VFSFile.xSectorSize() -> int Return the native underlying sector size. SQLite uses the value returned in determining the default database page size. If you do not implement the function or have an error then 4096 (the SQLite default) is returned. .. method:: VFSFile.xSync(flags: int) -> None Ensure data is on the disk platters (ie could survive a power failure immediately after the call returns) with the `sync flags `_ detailing what needs to be synced. .. method:: VFSFile.xTruncate(newsize: int) -> None Set the file length to *newsize* (which may be more or less than the current length). .. method:: VFSFile.xUnlock(level: int) -> None Decrease the lock to the level specified which is one of the `SQLITE_LOCK `_ family of constants. .. method:: VFSFile.xWrite(data: bytes, offset: int) -> None Write the *data* starting at absolute *offset*. You must write all the data requested, or return an error. If you have the file open for non-blocking I/O or if signals happen then it is possible for the underlying operating system to do a partial write. You will need to write the remaining data. :param offset: Where to start writing. URIFilename class ================= .. class:: URIFilename SQLite packs `uri parameters `__ and the filename together This class encapsulates that packing. The :ref:`example ` shows usage of this class. Your :meth:`VFS.xOpen` method will generally be passed one of these instead of a string as the filename if the URI flag was used or the main database flag is set. You can safely pass it on to the :class:`VFSFile` constructor which knows how to get the name back out. The URIFilename is only valid for the duration of the xOpen call. If you save and use the object later you will get an exception. .. method:: URIFilename.filename() -> str Returns the filename. .. index:: sqlite3_uri_key .. attribute:: URIFilename.parameters :type: tuple[str, ...] A tuple of the parameter names present. Calls: `sqlite3_uri_key `__ .. index:: sqlite3_uri_boolean .. method:: URIFilename.uri_boolean(name: str, default: bool) -> bool Returns the boolean value for parameter `name` or `default` if not present. Calls: `sqlite3_uri_boolean `__ .. index:: sqlite3_uri_int64 .. method:: URIFilename.uri_int(name: str, default: int) -> int Returns the integer value for parameter `name` or `default` if not present. Calls: `sqlite3_uri_int64 `__ .. index:: sqlite3_uri_parameter .. method:: URIFilename.uri_parameter(name: str) -> Optional[str] Returns the value of parameter `name` or None. Calls: `sqlite3_uri_parameter `__