|
Packit |
723767 |
NAME
|
|
Packit |
723767 |
DBD::SQLite - Self-contained RDBMS in a DBI Driver
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SYNOPSIS
|
|
Packit |
723767 |
use DBI;
|
|
Packit |
723767 |
my $dbh = DBI->connect("dbi:SQLite:dbname=$dbfile","","");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DESCRIPTION
|
|
Packit |
723767 |
SQLite is a public domain file-based relational database engine that you
|
|
Packit |
723767 |
can find at <http://www.sqlite.org/>.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DBD::SQLite is a Perl DBI driver for SQLite, that includes the entire
|
|
Packit |
723767 |
thing in the distribution. So in order to get a fast transaction capable
|
|
Packit |
723767 |
RDBMS working for your perl project you simply have to install this
|
|
Packit |
723767 |
module, and nothing else.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SQLite supports the following features:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Implements a large subset of SQL92
|
|
Packit |
723767 |
See <http://www.sqlite.org/lang.html> for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
A complete DB in a single disk file
|
|
Packit |
723767 |
Everything for your database is stored in a single disk file, making
|
|
Packit |
723767 |
it easier to move things around than with DBD::CSV.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Atomic commit and rollback
|
|
Packit |
723767 |
Yes, DBD::SQLite is small and light, but it supports full
|
|
Packit |
723767 |
transactions!
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Extensible
|
|
Packit |
723767 |
User-defined aggregate or regular functions can be registered with
|
|
Packit |
723767 |
the SQL parser.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
There's lots more to it, so please refer to the docs on the SQLite web
|
|
Packit |
723767 |
page, listed above, for SQL details. Also refer to DBI for details on
|
|
Packit |
723767 |
how to use DBI itself. The API works like every DBI module does.
|
|
Packit |
723767 |
However, currently many statement attributes are not implemented or are
|
|
Packit |
723767 |
limited by the typeless nature of the SQLite database.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
NOTABLE DIFFERENCES FROM OTHER DRIVERS
|
|
Packit |
723767 |
Database Name Is A File Name
|
|
Packit |
723767 |
SQLite creates a file per a database. You should pass the "path" of the
|
|
Packit |
723767 |
database file (with or without a parent directory) in the DBI connection
|
|
Packit |
723767 |
string (as a database "name"):
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $dbh = DBI->connect("dbi:SQLite:dbname=$dbfile","","");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The file is opened in read/write mode, and will be created if it does
|
|
Packit |
723767 |
not exist yet.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Although the database is stored in a single file, the directory
|
|
Packit |
723767 |
containing the database file must be writable by SQLite because the
|
|
Packit |
723767 |
library will create several temporary files there.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
If the filename $dbfile is ":memory:", then a private, temporary
|
|
Packit |
723767 |
in-memory database is created for the connection. This in-memory
|
|
Packit |
723767 |
database will vanish when the database connection is closed. It is handy
|
|
Packit |
723767 |
for your library tests.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Note that future versions of SQLite might make use of additional special
|
|
Packit |
723767 |
filenames that begin with the ":" character. It is recommended that when
|
|
Packit |
723767 |
a database filename actually does begin with a ":" character you should
|
|
Packit |
723767 |
prefix the filename with a pathname such as "./" to avoid ambiguity.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
If the filename $dbfile is an empty string, then a private, temporary
|
|
Packit |
723767 |
on-disk database will be created. This private database will be
|
|
Packit |
723767 |
automatically deleted as soon as the database connection is closed.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Accessing A Database With Other Tools
|
|
Packit |
723767 |
To access the database from the command line, try using "dbish" which
|
|
Packit |
723767 |
comes with the DBI::Shell module. Just type:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
dbish dbi:SQLite:foo.db
|
|
Packit |
723767 |
|
|
Packit |
723767 |
On the command line to access the file foo.db.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Alternatively you can install SQLite from the link above without
|
|
Packit |
723767 |
conflicting with DBD::SQLite and use the supplied "sqlite3" command line
|
|
Packit |
723767 |
tool.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Blobs
|
|
Packit |
723767 |
As of version 1.11, blobs should "just work" in SQLite as text columns.
|
|
Packit |
723767 |
However this will cause the data to be treated as a string, so SQL
|
|
Packit |
723767 |
statements such as length(x) will return the length of the column as a
|
|
Packit |
723767 |
NUL terminated string, rather than the size of the blob in bytes. In
|
|
Packit |
723767 |
order to store natively as a BLOB use the following code:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use DBI qw(:sql_types);
|
|
Packit |
723767 |
my $dbh = DBI->connect("dbi:SQLite:dbfile","","");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $blob = `cat foo.jpg`;
|
|
Packit |
723767 |
my $sth = $dbh->prepare("INSERT INTO mytable VALUES (1, ?)");
|
|
Packit |
723767 |
$sth->bind_param(1, $blob, SQL_BLOB);
|
|
Packit |
723767 |
$sth->execute();
|
|
Packit |
723767 |
|
|
Packit |
723767 |
And then retrieval just works:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$sth = $dbh->prepare("SELECT * FROM mytable WHERE id = 1");
|
|
Packit |
723767 |
$sth->execute();
|
|
Packit |
723767 |
my $row = $sth->fetch;
|
|
Packit |
723767 |
my $blobo = $row->[1];
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# now $blobo == $blob
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Functions And Bind Parameters
|
|
Packit |
723767 |
As of this writing, a SQL that compares a return value of a function
|
|
Packit |
723767 |
with a numeric bind value like this doesn't work as you might expect.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sth = $dbh->prepare(q{
|
|
Packit |
723767 |
SELECT bar FROM foo GROUP BY bar HAVING count(*) > ?;
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
$sth->execute(5);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This is because DBD::SQLite assumes that all the bind values are text
|
|
Packit |
723767 |
(and should be quoted) by default. Thus the above statement becomes like
|
|
Packit |
723767 |
this while executing:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT bar FROM foo GROUP BY bar HAVING count(*) > "5";
|
|
Packit |
723767 |
|
|
Packit |
723767 |
There are three workarounds for this.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Use bind_param() explicitly
|
|
Packit |
723767 |
As shown above in the "BLOB" section, you can always use
|
|
Packit |
723767 |
"bind_param()" to tell the type of a bind value.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use DBI qw(:sql_types); # Don't forget this
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sth = $dbh->prepare(q{
|
|
Packit |
723767 |
SELECT bar FROM foo GROUP BY bar HAVING count(*) > ?;
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
$sth->bind_param(1, 5, SQL_INTEGER);
|
|
Packit |
723767 |
$sth->execute();
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Add zero to make it a number
|
|
Packit |
723767 |
This is somewhat weird, but works anyway.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sth = $dbh->prepare(q{
|
|
Packit |
723767 |
SELECT bar FROM foo GROUP BY bar HAVING count(*) > (? + 0);
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
$sth->execute(5);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Set "sqlite_see_if_its_a_number" database handle attribute
|
|
Packit |
723767 |
As of version 1.32_02, you can use "sqlite_see_if_its_a_number" to
|
|
Packit |
723767 |
let DBD::SQLite to see if the bind values are numbers or not.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->{sqlite_see_if_its_a_number} = 1;
|
|
Packit |
723767 |
my $sth = $dbh->prepare(q{
|
|
Packit |
723767 |
SELECT bar FROM foo GROUP BY bar HAVING count(*) > ?;
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
$sth->execute(5);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
You can set it to true when you connect to a database.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $dbh = DBI->connect('dbi:SQLite:foo', undef, undef, {
|
|
Packit |
723767 |
AutoCommit => 1,
|
|
Packit |
723767 |
RaiseError => 1,
|
|
Packit |
723767 |
sqlite_see_if_its_a_number => 1,
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This is the most straightforward solution, but as noted above,
|
|
Packit |
723767 |
existing data in your databases created by DBD::SQLite have not
|
|
Packit |
723767 |
always been stored as numbers, so this *might* cause other obscure
|
|
Packit |
723767 |
problems. Use this sparingly when you handle existing databases. If
|
|
Packit |
723767 |
you handle databases created by other tools like native "sqlite3"
|
|
Packit |
723767 |
command line tool, this attribute would help you.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Placeholders
|
|
Packit |
723767 |
SQLite supports several placeholder expressions, including "?" and
|
|
Packit |
723767 |
":AAAA". Consult the DBI and sqlite documentation for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
<http://www.sqlite.org/lang_expr.html#varparam>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Note that a question mark actually means a next unused (numbered)
|
|
Packit |
723767 |
placeholder. You're advised not to use it with other (numbered or named)
|
|
Packit |
723767 |
placeholders to avoid confusion.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sth = $dbh->prepare(
|
|
Packit |
723767 |
'update TABLE set a=?1 where b=?2 and a IS NOT ?1'
|
|
Packit |
723767 |
);
|
|
Packit |
723767 |
$sth->execute(1, 2);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Foreign Keys
|
|
Packit |
723767 |
BE PREPARED! WOLVES APPROACH!!
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SQLite has started supporting foreign key constraints since 3.6.19
|
|
Packit |
723767 |
(released on Oct 14, 2009; bundled in DBD::SQLite 1.26_05). To be exact,
|
|
Packit |
723767 |
SQLite has long been able to parse a schema with foreign keys, but the
|
|
Packit |
723767 |
constraints has not been enforced. Now you can issue a pragma actually
|
|
Packit |
723767 |
to enable this feature and enforce the constraints.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
To do this, issue the following pragma (see below), preferably as soon
|
|
Packit |
723767 |
as you connect to a database and you're not in a transaction:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->do("PRAGMA foreign_keys = ON");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
And you can explicitly disable the feature whenever you like by turning
|
|
Packit |
723767 |
the pragma off:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->do("PRAGMA foreign_keys = OFF");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
As of this writing, this feature is disabled by default by the sqlite
|
|
Packit |
723767 |
team, and by us, to secure backward compatibility, as this feature may
|
|
Packit |
723767 |
break your applications, and actually broke some for us. If you have
|
|
Packit |
723767 |
used a schema with foreign key constraints but haven't cared them much
|
|
Packit |
723767 |
and supposed they're always ignored for SQLite, be prepared, and please
|
|
Packit |
723767 |
do extensive testing to ensure that your applications will continue to
|
|
Packit |
723767 |
work when the foreign keys support is enabled by default. It is very
|
|
Packit |
723767 |
likely that the sqlite team will turn it default-on in the future, and
|
|
Packit |
723767 |
we plan to do it NO LATER THAN they do so.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
See <http://www.sqlite.org/foreignkeys.html> for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Pragma
|
|
Packit |
723767 |
SQLite has a set of "Pragma"s to modifiy its operation or to query for
|
|
Packit |
723767 |
its internal data. These are specific to SQLite and are not likely to
|
|
Packit |
723767 |
work with other DBD libraries, but you may find some of these are quite
|
|
Packit |
723767 |
useful. DBD::SQLite actually sets some (like "show_datatypes") for you
|
|
Packit |
723767 |
when you connect to a database. See <http://www.sqlite.org/pragma.html>
|
|
Packit |
723767 |
for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Transactions
|
|
Packit |
723767 |
DBI/DBD::SQLite's transactions may be a bit confusing. They behave
|
|
Packit |
723767 |
differently according to the status of the "AutoCommit" flag:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
When the AutoCommit flag is on
|
|
Packit |
723767 |
You're supposed to always use the auto-commit mode, except you
|
|
Packit |
723767 |
explicitly begin a transaction, and when the transaction ended,
|
|
Packit |
723767 |
you're supposed to go back to the auto-commit mode. To begin a
|
|
Packit |
723767 |
transaction, call "begin_work" method, or issue a "BEGIN" statement.
|
|
Packit |
723767 |
To end it, call "commit/rollback" methods, or issue the
|
|
Packit |
723767 |
corresponding statements.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->{AutoCommit} = 1;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->begin_work; # or $dbh->do('BEGIN TRANSACTION');
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# $dbh->{AutoCommit} is turned off temporarily during a transaction;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->commit; # or $dbh->do('COMMIT');
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# $dbh->{AutoCommit} is turned on again;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
When the AutoCommit flag is off
|
|
Packit |
723767 |
You're supposed to always use the transactional mode, until you
|
|
Packit |
723767 |
explicitly turn on the AutoCommit flag. You can explicitly issue a
|
|
Packit |
723767 |
"BEGIN" statement (only when an actual transaction has not begun
|
|
Packit |
723767 |
yet) but you're not allowed to call "begin_work" method (if you
|
|
Packit |
723767 |
don't issue a "BEGIN", it will be issued internally). You can commit
|
|
Packit |
723767 |
or roll it back freely. Another transaction will automatically
|
|
Packit |
723767 |
begin if you execute another statement.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->{AutoCommit} = 0;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# $dbh->do('BEGIN TRANSACTION') is not necessary, but possible
|
|
Packit |
723767 |
|
|
Packit |
723767 |
...
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->commit; # or $dbh->do('COMMIT');
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# $dbh->{AutoCommit} stays intact;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->{AutoCommit} = 1; # ends the transactional mode
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This "AutoCommit" mode is independent from the autocommit mode of the
|
|
Packit |
723767 |
internal SQLite library, which always begins by a "BEGIN" statement, and
|
|
Packit |
723767 |
ends by a "COMMIT" or a <ROLLBACK>.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Transaction and Database Locking
|
|
Packit |
723767 |
Transaction by "AutoCommit" or "begin_work" is nice and handy, but
|
|
Packit |
723767 |
sometimes you may get an annoying "database is locked" error. This
|
|
Packit |
723767 |
typically happens when someone begins a transaction, and tries to write
|
|
Packit |
723767 |
to a database while other person is reading from the database (in
|
|
Packit |
723767 |
another transaction). You might be surprised but SQLite doesn't lock a
|
|
Packit |
723767 |
database when you just begin a normal (deferred) transaction to maximize
|
|
Packit |
723767 |
concurrency. It reserves a lock when you issue a statement to write, but
|
|
Packit |
723767 |
until you actually try to write with a "commit" statement, it allows
|
|
Packit |
723767 |
other people to read from the database. However, reading from the
|
|
Packit |
723767 |
database also requires "shared lock", and that prevents to give you the
|
|
Packit |
723767 |
"exclusive lock" you reserved, thus you get the "database is locked"
|
|
Packit |
723767 |
error, and other people will get the same error if they try to write
|
|
Packit |
723767 |
afterwards, as you still have a "pending" lock. "busy_timeout" doesn't
|
|
Packit |
723767 |
help in this case.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
To avoid this, set a transaction type explicitly. You can issue a "begin
|
|
Packit |
723767 |
immediate transaction" (or "begin exclusive transaction") for each
|
|
Packit |
723767 |
transaction, or set "sqlite_use_immediate_transaction" database handle
|
|
Packit |
723767 |
attribute to true (since 1.30_02) to always use an immediate transaction
|
|
Packit |
723767 |
(even when you simply use "begin_work" or turn off the "AutoCommit".).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $dbh = DBI->connect("dbi:SQLite::memory:", "", "", {
|
|
Packit |
723767 |
sqlite_use_immediate_transaction => 1,
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Note that this works only when all of the connections use the same
|
|
Packit |
723767 |
(non-deferred) transaction. See <http://sqlite.org/lockingv3.html> for
|
|
Packit |
723767 |
locking details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
"$sth->finish" and Transaction Rollback
|
|
Packit |
723767 |
As the DBI doc says, you almost certainly do not need to call "finish"
|
|
Packit |
723767 |
in DBI method if you fetch all rows (probably in a loop). However, there
|
|
Packit |
723767 |
are several exceptions to this rule, and rolling-back of an unfinished
|
|
Packit |
723767 |
"SELECT" statement is one of such exceptional cases.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SQLite prohibits "ROLLBACK" of unfinished "SELECT" statements in a
|
|
Packit |
723767 |
transaction (See <http://sqlite.org/lang_transaction.html> for details).
|
|
Packit |
723767 |
So you need to call "finish" before you issue a rollback.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$sth = $dbh->prepare("SELECT * FROM t");
|
|
Packit |
723767 |
$dbh->begin_work;
|
|
Packit |
723767 |
eval {
|
|
Packit |
723767 |
$sth->execute;
|
|
Packit |
723767 |
$row = $sth->fetch;
|
|
Packit |
723767 |
...
|
|
Packit |
723767 |
die "For some reason";
|
|
Packit |
723767 |
...
|
|
Packit |
723767 |
};
|
|
Packit |
723767 |
if($@) {
|
|
Packit |
723767 |
$sth->finish; # You need this for SQLite
|
|
Packit |
723767 |
$dbh->rollback;
|
|
Packit |
723767 |
} else {
|
|
Packit |
723767 |
$dbh->commit;
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Processing Multiple Statements At A Time
|
|
Packit |
723767 |
DBI's statement handle is not supposed to process multiple statements at
|
|
Packit |
723767 |
a time. So if you pass a string that contains multiple statements (a
|
|
Packit |
723767 |
"dump") to a statement handle (via "prepare" or "do"), DBD::SQLite only
|
|
Packit |
723767 |
processes the first statement, and discards the rest.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Since 1.30_01, you can retrieve those ignored (unprepared) statements
|
|
Packit |
723767 |
via "$sth->{sqlite_unprepared_statements}". It usually contains nothing
|
|
Packit |
723767 |
but white spaces, but if you really care, you can check this attribute
|
|
Packit |
723767 |
to see if there's anything left undone. Also, if you set a
|
|
Packit |
723767 |
"sqlite_allow_multiple_statements" attribute of a database handle to
|
|
Packit |
723767 |
true when you connect to a database, "do" method automatically checks
|
|
Packit |
723767 |
the "sqlite_unprepared_statements" attribute, and if it finds anything
|
|
Packit |
723767 |
undone (even if what's left is just a single white space), it repeats
|
|
Packit |
723767 |
the process again, to the end.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Performance
|
|
Packit |
723767 |
SQLite is fast, very fast. Matt processed his 72MB log file with it,
|
|
Packit |
723767 |
inserting the data (400,000+ rows) by using transactions and only
|
|
Packit |
723767 |
committing every 1000 rows (otherwise the insertion is quite slow), and
|
|
Packit |
723767 |
then performing queries on the data.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Queries like count(*) and avg(bytes) took fractions of a second to
|
|
Packit |
723767 |
return, but what surprised him most of all was:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT url, count(*) as count
|
|
Packit |
723767 |
FROM access_log
|
|
Packit |
723767 |
GROUP BY url
|
|
Packit |
723767 |
ORDER BY count desc
|
|
Packit |
723767 |
LIMIT 20
|
|
Packit |
723767 |
|
|
Packit |
723767 |
To discover the top 20 hit URLs on the site (<http://axkit.org>), and it
|
|
Packit |
723767 |
returned within 2 seconds. He was seriously considering switching his
|
|
Packit |
723767 |
log analysis code to use this little speed demon!
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Oh yeah, and that was with no indexes on the table, on a 400MHz PIII.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
For best performance be sure to tune your hdparm settings if you are
|
|
Packit |
723767 |
using linux. Also you might want to set:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
PRAGMA synchronous = OFF
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Which will prevent sqlite from doing fsync's when writing (which slows
|
|
Packit |
723767 |
down non-transactional writes significantly) at the expense of some
|
|
Packit |
723767 |
peace of mind. Also try playing with the cache_size pragma.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The memory usage of SQLite can also be tuned using the cache_size
|
|
Packit |
723767 |
pragma.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->do("PRAGMA cache_size = 800000");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The above will allocate 800M for DB cache; the default is 2M. Your sweet
|
|
Packit |
723767 |
spot probably lies somewhere in between.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DRIVER PRIVATE ATTRIBUTES
|
|
Packit |
723767 |
Database Handle Attributes
|
|
Packit |
723767 |
sqlite_version
|
|
Packit |
723767 |
Returns the version of the SQLite library which DBD::SQLite is
|
|
Packit |
723767 |
using, e.g., "2.8.0". Can only be read.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sqlite_unicode
|
|
Packit |
723767 |
If set to a true value, DBD::SQLite will turn the UTF-8 flag on for
|
|
Packit |
723767 |
all text strings coming out of the database (this feature is
|
|
Packit |
723767 |
currently disabled for perl < 5.8.5). For more details on the UTF-8
|
|
Packit |
723767 |
flag see perlunicode. The default is for the UTF-8 flag to be turned
|
|
Packit |
723767 |
off.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Also note that due to some bizarreness in SQLite's type system (see
|
|
Packit |
723767 |
<http://www.sqlite.org/datatype3.html>), if you want to retain
|
|
Packit |
723767 |
blob-style behavior for some columns under "$dbh->{sqlite_unicode} =
|
|
Packit |
723767 |
1" (say, to store images in the database), you have to state so
|
|
Packit |
723767 |
explicitly using the 3-argument form of "bind_param" in DBI when
|
|
Packit |
723767 |
doing updates:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use DBI qw(:sql_types);
|
|
Packit |
723767 |
$dbh->{sqlite_unicode} = 1;
|
|
Packit |
723767 |
my $sth = $dbh->prepare("INSERT INTO mytable (blobcolumn) VALUES (?)");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# Binary_data will be stored as is.
|
|
Packit |
723767 |
$sth->bind_param(1, $binary_data, SQL_BLOB);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Defining the column type as "BLOB" in the DDL is not sufficient.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This attribute was originally named as "unicode", and renamed to
|
|
Packit |
723767 |
"sqlite_unicode" for integrity since version 1.26_06. Old "unicode"
|
|
Packit |
723767 |
attribute is still accessible but will be deprecated in the near
|
|
Packit |
723767 |
future.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sqlite_allow_multiple_statements
|
|
Packit |
723767 |
If you set this to true, "do" method will process multiple
|
|
Packit |
723767 |
statements at one go. This may be handy, but with performance
|
|
Packit |
723767 |
penalty. See above for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sqlite_use_immediate_transaction
|
|
Packit |
723767 |
If you set this to true, DBD::SQLite tries to issue a "begin
|
|
Packit |
723767 |
immediate transaction" (instead of "begin transaction") when
|
|
Packit |
723767 |
necessary. See above for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sqlite_see_if_its_a_number
|
|
Packit |
723767 |
If you set this to true, DBD::SQLite tries to see if the bind values
|
|
Packit |
723767 |
are number or not, and does not quote if they are numbers. See above
|
|
Packit |
723767 |
for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Statement Handle Attributes
|
|
Packit |
723767 |
sqlite_unprepared_statements
|
|
Packit |
723767 |
Returns an unprepared part of the statement you pass to "prepare".
|
|
Packit |
723767 |
Typically this contains nothing but white spaces after a semicolon.
|
|
Packit |
723767 |
See above for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
METHODS
|
|
Packit |
723767 |
See also to the DBI documentation for the details of other common
|
|
Packit |
723767 |
methods.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
table_info
|
|
Packit |
723767 |
$sth = $dbh->table_info(undef, $schema, $table, $type, \%attr);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Returns all tables and schemas (databases) as specified in "table_info"
|
|
Packit |
723767 |
in DBI. The schema and table arguments will do a "LIKE" search. You can
|
|
Packit |
723767 |
specify an ESCAPE character by including an 'Escape' attribute in
|
|
Packit |
723767 |
\%attr. The $type argument accepts a comma separated list of the
|
|
Packit |
723767 |
following types 'TABLE', 'VIEW', 'LOCAL TEMPORARY' and 'SYSTEM TABLE'
|
|
Packit |
723767 |
(by default all are returned). Note that a statement handle is returned,
|
|
Packit |
723767 |
and not a direct list of tables.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The following fields are returned:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
TABLE_CAT: Always NULL, as SQLite does not have the concept of catalogs.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
TABLE_SCHEM: The name of the schema (database) that the table or view is
|
|
Packit |
723767 |
in. The default schema is 'main', temporary tables are in 'temp' and
|
|
Packit |
723767 |
other databases will be in the name given when the database was
|
|
Packit |
723767 |
attached.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
TABLE_NAME: The name of the table or view.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
TABLE_TYPE: The type of object returned. Will be one of 'TABLE', 'VIEW',
|
|
Packit |
723767 |
'LOCAL TEMPORARY' or 'SYSTEM TABLE'.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
primary_key, primary_key_info
|
|
Packit |
723767 |
@names = $dbh->primary_key(undef, $schema, $table);
|
|
Packit |
723767 |
$sth = $dbh->primary_key_info(undef, $schema, $table, \%attr);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
You can retrieve primary key names or more detailed information. As
|
|
Packit |
723767 |
noted above, SQLite does not have the concept of catalogs, so the first
|
|
Packit |
723767 |
argument of the mothods is usually "undef", and you'll usually set
|
|
Packit |
723767 |
"undef" for the second one (unless you want to know the primary keys of
|
|
Packit |
723767 |
temporary tables).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DRIVER PRIVATE METHODS
|
|
Packit |
723767 |
The following methods can be called via the func() method with a little
|
|
Packit |
723767 |
tweak, but the use of func() method is now discouraged by the DBI author
|
|
Packit |
723767 |
for various reasons (see DBI's document
|
|
Packit |
723767 |
|
|
Packit |
723767 |
o_expose_driver-private_methods> for details). So, if you're using DBI
|
|
Packit |
723767 |
>= 1.608, use these "sqlite_" methods. If you need to use an older DBI,
|
|
Packit |
723767 |
you can call these like this:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->func( ..., "(method name without sqlite_ prefix)" );
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Exception: "sqlite_trace" should always be called as is, even with
|
|
Packit |
723767 |
"func()" method (to avoid conflict with DBI's trace() method).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->func( ..., "sqlite_trace");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_last_insert_rowid()
|
|
Packit |
723767 |
This method returns the last inserted rowid. If you specify an INTEGER
|
|
Packit |
723767 |
PRIMARY KEY as the first column in your table, that is the column that
|
|
Packit |
723767 |
is returned. Otherwise, it is the hidden ROWID column. See the sqlite
|
|
Packit |
723767 |
docs for details.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Generally you should not be using this method. Use the DBI
|
|
Packit |
723767 |
last_insert_id method instead. The usage of this is:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$h->last_insert_id($catalog, $schema, $table_name, $field_name [, \%attr ])
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Running "$h->last_insert_id("","","","")" is the equivalent of running
|
|
Packit |
723767 |
"$dbh->sqlite_last_insert_rowid()" directly.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_busy_timeout()
|
|
Packit |
723767 |
Retrieve the current busy timeout.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_busy_timeout( $ms )
|
|
Packit |
723767 |
Set the current busy timeout. The timeout is in milliseconds.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_create_function( $name, $argc, $code_ref )
|
|
Packit |
723767 |
This method will register a new function which will be usable in an SQL
|
|
Packit |
723767 |
query. The method's parameters are:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$name
|
|
Packit |
723767 |
The name of the function. This is the name of the function as it
|
|
Packit |
723767 |
will be used from SQL.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$argc
|
|
Packit |
723767 |
The number of arguments taken by the function. If this number is -1,
|
|
Packit |
723767 |
the function can take any number of arguments.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref
|
|
Packit |
723767 |
This should be a reference to the function's implementation.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
For example, here is how to define a now() function which returns the
|
|
Packit |
723767 |
current number of seconds since the epoch:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_create_function( 'now', 0, sub { return time } );
|
|
Packit |
723767 |
|
|
Packit |
723767 |
After this, it could be use from SQL as:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
INSERT INTO mytable ( now() );
|
|
Packit |
723767 |
|
|
Packit |
723767 |
REGEXP function
|
|
Packit |
723767 |
SQLite includes syntactic support for an infix operator 'REGEXP', but
|
|
Packit |
723767 |
without any implementation. The "DBD::SQLite" driver automatically
|
|
Packit |
723767 |
registers an implementation that performs standard perl regular
|
|
Packit |
723767 |
expression matching, using current locale. So for example you can search
|
|
Packit |
723767 |
for words starting with an 'A' with a query like
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT * from table WHERE column REGEXP '\bA\w+'
|
|
Packit |
723767 |
|
|
Packit |
723767 |
If you want case-insensitive searching, use perl regex flags, like this
|
|
Packit |
723767 |
:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT * from table WHERE column REGEXP '(?i:\bA\w+)'
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The default REGEXP implementation can be overridden through the
|
|
Packit |
723767 |
"create_function" API described above.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Note that regexp matching will not use SQLite indices, but will iterate
|
|
Packit |
723767 |
over all rows, so it could be quite costly in terms of performance.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_create_collation( $name, $code_ref )
|
|
Packit |
723767 |
This method manually registers a new function which will be usable in an
|
|
Packit |
723767 |
SQL query as a COLLATE option for sorting. Such functions can also be
|
|
Packit |
723767 |
registered automatically on demand: see section "COLLATION FUNCTIONS"
|
|
Packit |
723767 |
below.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The method's parameters are:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$name
|
|
Packit |
723767 |
The name of the function exposed to SQL.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref
|
|
Packit |
723767 |
Reference to the function's implementation. The driver will check
|
|
Packit |
723767 |
that this is a proper sorting function.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_collation_needed( $code_ref )
|
|
Packit |
723767 |
This method manually registers a callback function that will be invoked
|
|
Packit |
723767 |
whenever an undefined collation sequence is required from an SQL
|
|
Packit |
723767 |
statement. The callback is invoked as
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref->($dbh, $collation_name)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
and should register the desired collation using
|
|
Packit |
723767 |
"sqlite_create_collation".
|
|
Packit |
723767 |
|
|
Packit |
723767 |
An initial callback is already registered by "DBD::SQLite", so for most
|
|
Packit |
723767 |
common cases it will be simpler to just add your collation sequences in
|
|
Packit |
723767 |
the %DBD::SQLite::COLLATION hash (see section "COLLATION FUNCTIONS"
|
|
Packit |
723767 |
below).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_create_aggregate( $name, $argc, $pkg )
|
|
Packit |
723767 |
This method will register a new aggregate function which can then be
|
|
Packit |
723767 |
used from SQL. The method's parameters are:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$name
|
|
Packit |
723767 |
The name of the aggregate function, this is the name under which the
|
|
Packit |
723767 |
function will be available from SQL.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$argc
|
|
Packit |
723767 |
This is an integer which tells the SQL parser how many arguments the
|
|
Packit |
723767 |
function takes. If that number is -1, the function can take any
|
|
Packit |
723767 |
number of arguments.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$pkg
|
|
Packit |
723767 |
This is the package which implements the aggregator interface.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The aggregator interface consists of defining three methods:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
new()
|
|
Packit |
723767 |
This method will be called once to create an object which should be
|
|
Packit |
723767 |
used to aggregate the rows in a particular group. The step() and
|
|
Packit |
723767 |
finalize() methods will be called upon the reference return by the
|
|
Packit |
723767 |
method.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
step(@_)
|
|
Packit |
723767 |
This method will be called once for each row in the aggregate.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
finalize()
|
|
Packit |
723767 |
This method will be called once all rows in the aggregate were
|
|
Packit |
723767 |
processed and it should return the aggregate function's result. When
|
|
Packit |
723767 |
there is no rows in the aggregate, finalize() will be called right
|
|
Packit |
723767 |
after new().
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Here is a simple aggregate function which returns the variance (example
|
|
Packit |
723767 |
adapted from pysqlite):
|
|
Packit |
723767 |
|
|
Packit |
723767 |
package variance;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sub new { bless [], shift; }
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sub step {
|
|
Packit |
723767 |
my ( $self, $value ) = @_;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
push @$self, $value;
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sub finalize {
|
|
Packit |
723767 |
my $self = $_[0];
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $n = @$self;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# Variance is NULL unless there is more than one row
|
|
Packit |
723767 |
return undef unless $n || $n == 1;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $mu = 0;
|
|
Packit |
723767 |
foreach my $v ( @$self ) {
|
|
Packit |
723767 |
$mu += $v;
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
$mu /= $n;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sigma = 0;
|
|
Packit |
723767 |
foreach my $v ( @$self ) {
|
|
Packit |
723767 |
$sigma += ($x - $mu)**2;
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
$sigma = $sigma / ($n - 1);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
return $sigma;
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_create_aggregate( "variance", 1, 'variance' );
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The aggregate function can then be used as:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT group_name, variance(score)
|
|
Packit |
723767 |
FROM results
|
|
Packit |
723767 |
GROUP BY group_name;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
For more examples, see the DBD::SQLite::Cookbook.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_progress_handler( $n_opcodes, $code_ref )
|
|
Packit |
723767 |
This method registers a handler to be invoked periodically during long
|
|
Packit |
723767 |
running calls to SQLite.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
An example use for this interface is to keep a GUI updated during a
|
|
Packit |
723767 |
large query. The parameters are:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$n_opcodes
|
|
Packit |
723767 |
The progress handler is invoked once for every $n_opcodes virtual
|
|
Packit |
723767 |
machine opcodes in SQLite.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref
|
|
Packit |
723767 |
Reference to the handler subroutine. If the progress handler returns
|
|
Packit |
723767 |
non-zero, the SQLite operation is interrupted. This feature can be
|
|
Packit |
723767 |
used to implement a "Cancel" button on a GUI dialog box.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Set this argument to "undef" if you want to unregister a previous
|
|
Packit |
723767 |
progress handler.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_commit_hook( $code_ref )
|
|
Packit |
723767 |
This method registers a callback function to be invoked whenever a
|
|
Packit |
723767 |
transaction is committed. Any callback set by a previous call to
|
|
Packit |
723767 |
"sqlite_commit_hook" is overridden. A reference to the previous callback
|
|
Packit |
723767 |
(if any) is returned. Registering an "undef" disables the callback.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
When the commit hook callback returns zero, the commit operation is
|
|
Packit |
723767 |
allowed to continue normally. If the callback returns non-zero, then the
|
|
Packit |
723767 |
commit is converted into a rollback (in that case, any attempt to
|
|
Packit |
723767 |
*explicitly* call "$dbh->rollback()" afterwards would yield an error).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_rollback_hook( $code_ref )
|
|
Packit |
723767 |
This method registers a callback function to be invoked whenever a
|
|
Packit |
723767 |
transaction is rolled back. Any callback set by a previous call to
|
|
Packit |
723767 |
"sqlite_rollback_hook" is overridden. A reference to the previous
|
|
Packit |
723767 |
callback (if any) is returned. Registering an "undef" disables the
|
|
Packit |
723767 |
callback.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_update_hook( $code_ref )
|
|
Packit |
723767 |
This method registers a callback function to be invoked whenever a row
|
|
Packit |
723767 |
is updated, inserted or deleted. Any callback set by a previous call to
|
|
Packit |
723767 |
"sqlite_update_hook" is overridden. A reference to the previous callback
|
|
Packit |
723767 |
(if any) is returned. Registering an "undef" disables the callback.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The callback will be called as
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref->($action_code, $database, $table, $rowid)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$action_code
|
|
Packit |
723767 |
is an integer equal to either "DBD::SQLite::INSERT",
|
|
Packit |
723767 |
"DBD::SQLite::DELETE" or "DBD::SQLite::UPDATE" (see "Action Codes");
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$database
|
|
Packit |
723767 |
is the name of the database containing the affected row;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$table
|
|
Packit |
723767 |
is the name of the table containing the affected row;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$rowid
|
|
Packit |
723767 |
is the unique 64-bit signed integer key of the affected row within
|
|
Packit |
723767 |
that table.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_set_authorizer( $code_ref )
|
|
Packit |
723767 |
This method registers an authorizer callback to be invoked whenever SQL
|
|
Packit |
723767 |
statements are being compiled by the "prepare" in DBI method. The
|
|
Packit |
723767 |
authorizer callback should return "DBD::SQLite::OK" to allow the action,
|
|
Packit |
723767 |
"DBD::SQLite::IGNORE" to disallow the specific action but allow the SQL
|
|
Packit |
723767 |
statement to continue to be compiled, or "DBD::SQLite::DENY" to cause
|
|
Packit |
723767 |
the entire SQL statement to be rejected with an error. If the authorizer
|
|
Packit |
723767 |
callback returns any other value, then then "prepare" call that
|
|
Packit |
723767 |
triggered the authorizer will fail with an error message.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
An authorizer is used when preparing SQL statements from an untrusted
|
|
Packit |
723767 |
source, to ensure that the SQL statements do not try to access data they
|
|
Packit |
723767 |
are not allowed to see, or that they do not try to execute malicious
|
|
Packit |
723767 |
statements that damage the database. For example, an application may
|
|
Packit |
723767 |
allow a user to enter arbitrary SQL queries for evaluation by a
|
|
Packit |
723767 |
database. But the application does not want the user to be able to make
|
|
Packit |
723767 |
arbitrary changes to the database. An authorizer could then be put in
|
|
Packit |
723767 |
place while the user-entered SQL is being prepared that disallows
|
|
Packit |
723767 |
everything except SELECT statements.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The callback will be called as
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref->($action_code, $string1, $string2, $database, $trigger_or_view)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$action_code
|
|
Packit |
723767 |
is an integer that specifies what action is being authorized (see
|
|
Packit |
723767 |
"Action Codes").
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$string1, $string2
|
|
Packit |
723767 |
are strings that depend on the action code (see "Action Codes").
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$database
|
|
Packit |
723767 |
is the name of the database ("main", "temp", etc.) if applicable.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$trigger_or_view
|
|
Packit |
723767 |
is the name of the inner-most trigger or view that is responsible
|
|
Packit |
723767 |
for the access attempt, or "undef" if this access attempt is
|
|
Packit |
723767 |
directly from top-level SQL code.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_backup_from_file( $filename )
|
|
Packit |
723767 |
This method accesses the SQLite Online Backup API, and will take a
|
|
Packit |
723767 |
backup of the named database file, copying it to, and overwriting, your
|
|
Packit |
723767 |
current database connection. This can be particularly handy if your
|
|
Packit |
723767 |
current connection is to the special :memory: database, and you wish to
|
|
Packit |
723767 |
populate it from an existing DB.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_backup_to_file( $filename )
|
|
Packit |
723767 |
This method accesses the SQLite Online Backup API, and will take a
|
|
Packit |
723767 |
backup of the currently connected database, and write it out to the
|
|
Packit |
723767 |
named file.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_enable_load_extension( $bool )
|
|
Packit |
723767 |
Calling this method with a true value enables loading (external) sqlite3
|
|
Packit |
723767 |
extensions. After the call, you can load extensions like this:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_enable_load_extension(1);
|
|
Packit |
723767 |
$sth = $dbh->prepare("select load_extension('libsqlitefunctions.so')")
|
|
Packit |
723767 |
or die "Cannot prepare: " . $dbh->errstr();
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_trace( $code_ref )
|
|
Packit |
723767 |
This method registers a trace callback to be invoked whenever SQL
|
|
Packit |
723767 |
statements are being run.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The callback will be called as
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref->($statement)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$statement
|
|
Packit |
723767 |
is a UTF-8 rendering of the SQL statement text as the statement
|
|
Packit |
723767 |
first begins executing.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Additional callbacks might occur as each triggered subprogram is
|
|
Packit |
723767 |
entered. The callbacks for triggers contain a UTF-8 SQL comment that
|
|
Packit |
723767 |
identifies the trigger.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
See also "TRACING" in DBI for better tracing options.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->sqlite_profile( $code_ref )
|
|
Packit |
723767 |
This method registers a profile callback to be invoked whenever a SQL
|
|
Packit |
723767 |
statement finishes.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The callback will be called as
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$code_ref->($statement, $elapsed_time)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$statement
|
|
Packit |
723767 |
is the original statement text (without bind parameters).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$elapsed_time
|
|
Packit |
723767 |
is an estimate of wall-clock time of how long that statement took to
|
|
Packit |
723767 |
run (in milliseconds).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This method is considered experimental and is subject to change in
|
|
Packit |
723767 |
future versions of SQLite.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
See also DBI::Profile for better profiling options.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DBD::SQLite::compile_options()
|
|
Packit |
723767 |
Returns an array of compile options (available since sqlite 3.6.23,
|
|
Packit |
723767 |
bundled in DBD::SQLite 1.30_01), or an empty array if the bundled
|
|
Packit |
723767 |
library is old or compiled with SQLITE_OMIT_COMPILEOPTION_DIAGS.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DRIVER CONSTANTS
|
|
Packit |
723767 |
A subset of SQLite C constants are made available to Perl, because they
|
|
Packit |
723767 |
may be needed when writing hooks or authorizer callbacks. For accessing
|
|
Packit |
723767 |
such constants, the "DBD::SQLite" module must be explicitly "use"d at
|
|
Packit |
723767 |
compile time. For example, an authorizer that forbids any DELETE
|
|
Packit |
723767 |
operation would be written as follows :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use DBD::SQLite;
|
|
Packit |
723767 |
$dbh->sqlite_set_authorizer(sub {
|
|
Packit |
723767 |
my $action_code = shift;
|
|
Packit |
723767 |
return $action_code == DBD::SQLite::DELETE ? DBD::SQLite::DENY
|
|
Packit |
723767 |
: DBD::SQLite::OK;
|
|
Packit |
723767 |
});
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The list of constants implemented in "DBD::SQLite" is given below; more
|
|
Packit |
723767 |
information can be found ad at
|
|
Packit |
723767 |
<http://www.sqlite.org/c3ref/constlist.html>.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Authorizer Return Codes
|
|
Packit |
723767 |
OK
|
|
Packit |
723767 |
DENY
|
|
Packit |
723767 |
IGNORE
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Action Codes
|
|
Packit |
723767 |
The "set_authorizer" method registers a callback function that is
|
|
Packit |
723767 |
invoked to authorize certain SQL statement actions. The first parameter
|
|
Packit |
723767 |
to the callback is an integer code that specifies what action is being
|
|
Packit |
723767 |
authorized. The second and third parameters to the callback are strings,
|
|
Packit |
723767 |
the meaning of which varies according to the action code. Below is the
|
|
Packit |
723767 |
list of action codes, together with their associated strings.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# constant string1 string2
|
|
Packit |
723767 |
# ======== ======= =======
|
|
Packit |
723767 |
CREATE_INDEX Index Name Table Name
|
|
Packit |
723767 |
CREATE_TABLE Table Name undef
|
|
Packit |
723767 |
CREATE_TEMP_INDEX Index Name Table Name
|
|
Packit |
723767 |
CREATE_TEMP_TABLE Table Name undef
|
|
Packit |
723767 |
CREATE_TEMP_TRIGGER Trigger Name Table Name
|
|
Packit |
723767 |
CREATE_TEMP_VIEW View Name undef
|
|
Packit |
723767 |
CREATE_TRIGGER Trigger Name Table Name
|
|
Packit |
723767 |
CREATE_VIEW View Name undef
|
|
Packit |
723767 |
DELETE Table Name undef
|
|
Packit |
723767 |
DROP_INDEX Index Name Table Name
|
|
Packit |
723767 |
DROP_TABLE Table Name undef
|
|
Packit |
723767 |
DROP_TEMP_INDEX Index Name Table Name
|
|
Packit |
723767 |
DROP_TEMP_TABLE Table Name undef
|
|
Packit |
723767 |
DROP_TEMP_TRIGGER Trigger Name Table Name
|
|
Packit |
723767 |
DROP_TEMP_VIEW View Name undef
|
|
Packit |
723767 |
DROP_TRIGGER Trigger Name Table Name
|
|
Packit |
723767 |
DROP_VIEW View Name undef
|
|
Packit |
723767 |
INSERT Table Name undef
|
|
Packit |
723767 |
PRAGMA Pragma Name 1st arg or undef
|
|
Packit |
723767 |
READ Table Name Column Name
|
|
Packit |
723767 |
SELECT undef undef
|
|
Packit |
723767 |
TRANSACTION Operation undef
|
|
Packit |
723767 |
UPDATE Table Name Column Name
|
|
Packit |
723767 |
ATTACH Filename undef
|
|
Packit |
723767 |
DETACH Database Name undef
|
|
Packit |
723767 |
ALTER_TABLE Database Name Table Name
|
|
Packit |
723767 |
REINDEX Index Name undef
|
|
Packit |
723767 |
ANALYZE Table Name undef
|
|
Packit |
723767 |
CREATE_VTABLE Table Name Module Name
|
|
Packit |
723767 |
DROP_VTABLE Table Name Module Name
|
|
Packit |
723767 |
FUNCTION undef Function Name
|
|
Packit |
723767 |
SAVEPOINT Operation Savepoint Name
|
|
Packit |
723767 |
|
|
Packit |
723767 |
COLLATION FUNCTIONS
|
|
Packit |
723767 |
Definition
|
|
Packit |
723767 |
SQLite v3 provides the ability for users to supply arbitrary comparison
|
|
Packit |
723767 |
functions, known as user-defined "collation sequences" or "collating
|
|
Packit |
723767 |
functions", to be used for comparing two text values.
|
|
Packit |
723767 |
<http://www.sqlite.org/datatype3.html#collation> explains how collations
|
|
Packit |
723767 |
are used in various SQL expressions.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Builtin collation sequences
|
|
Packit |
723767 |
The following collation sequences are builtin within SQLite :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
BINARY
|
|
Packit |
723767 |
Compares string data using memcmp(), regardless of text encoding.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
NOCASE
|
|
Packit |
723767 |
The same as binary, except the 26 upper case characters of ASCII are
|
|
Packit |
723767 |
folded to their lower case equivalents before the comparison is
|
|
Packit |
723767 |
performed. Note that only ASCII characters are case folded. SQLite
|
|
Packit |
723767 |
does not attempt to do full UTF case folding due to the size of the
|
|
Packit |
723767 |
tables required.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
RTRIM
|
|
Packit |
723767 |
The same as binary, except that trailing space characters are
|
|
Packit |
723767 |
ignored.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
In addition, "DBD::SQLite" automatically installs the following
|
|
Packit |
723767 |
collation sequences :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
perl
|
|
Packit |
723767 |
corresponds to the Perl "cmp" operator
|
|
Packit |
723767 |
|
|
Packit |
723767 |
perllocale
|
|
Packit |
723767 |
Perl "cmp" operator, in a context where "use locale" is activated.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Usage
|
|
Packit |
723767 |
You can write for example
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE TABLE foo(
|
|
Packit |
723767 |
txt1 COLLATE perl,
|
|
Packit |
723767 |
txt2 COLLATE perllocale,
|
|
Packit |
723767 |
txt3 COLLATE nocase
|
|
Packit |
723767 |
)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
or
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SELECT * FROM foo ORDER BY name COLLATE perllocale
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Unicode handling
|
|
Packit |
723767 |
If the attribute "$dbh->{sqlite_unicode}" is set, strings coming from
|
|
Packit |
723767 |
the database and passed to the collation function will be properly
|
|
Packit |
723767 |
tagged with the utf8 flag; but this only works if the "sqlite_unicode"
|
|
Packit |
723767 |
attribute is set before the first call to a perl collation sequence .
|
|
Packit |
723767 |
The recommended way to activate unicode is to set the parameter at
|
|
Packit |
723767 |
connection time :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $dbh = DBI->connect(
|
|
Packit |
723767 |
"dbi:SQLite:dbname=foo", "", "",
|
|
Packit |
723767 |
{
|
|
Packit |
723767 |
RaiseError => 1,
|
|
Packit |
723767 |
sqlite_unicode => 1,
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Adding user-defined collations
|
|
Packit |
723767 |
The native SQLite API for adding user-defined collations is exposed
|
|
Packit |
723767 |
through methods "sqlite_create_collation" and "sqlite_collation_needed".
|
|
Packit |
723767 |
|
|
Packit |
723767 |
To avoid calling these functions every time a $dbh handle is created,
|
|
Packit |
723767 |
"DBD::SQLite" offers a simpler interface through the
|
|
Packit |
723767 |
%DBD::SQLite::COLLATION hash : just insert your own collation functions
|
|
Packit |
723767 |
in that hash, and whenever an unknown collation name is encountered in
|
|
Packit |
723767 |
SQL, the appropriate collation function will be loaded on demand from
|
|
Packit |
723767 |
the hash. For example, here is a way to sort text values regardless of
|
|
Packit |
723767 |
their accented characters :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use DBD::SQLite;
|
|
Packit |
723767 |
$DBD::SQLite::COLLATION{no_accents} = sub {
|
|
Packit |
723767 |
my ( $a, $b ) = map lc, @_;
|
|
Packit |
723767 |
tr[????????????????????????????]
|
|
Packit |
723767 |
[aaaaaacdeeeeiiiinoooooouuuuy] for $a, $b;
|
|
Packit |
723767 |
$a cmp $b;
|
|
Packit |
723767 |
};
|
|
Packit |
723767 |
my $dbh = DBI->connect("dbi:SQLite:dbname=dbfile");
|
|
Packit |
723767 |
my $sql = "SELECT ... FROM ... ORDER BY ... COLLATE no_accents");
|
|
Packit |
723767 |
my $rows = $dbh->selectall_arrayref($sql);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The builtin "perl" or "perllocale" collations are predefined in that
|
|
Packit |
723767 |
same hash.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The COLLATION hash is a global registry within the current process;
|
|
Packit |
723767 |
hence there is a risk of undesired side-effects. Therefore, to prevent
|
|
Packit |
723767 |
action at distance, the hash is implemented as a "write-only" hash, that
|
|
Packit |
723767 |
will happily accept new entries, but will raise an exception if any
|
|
Packit |
723767 |
attempt is made to override or delete a existing entry (including the
|
|
Packit |
723767 |
builtin "perl" and "perllocale").
|
|
Packit |
723767 |
|
|
Packit |
723767 |
If you really, really need to change or delete an entry, you can always
|
|
Packit |
723767 |
grab the tied object underneath %DBD::SQLite::COLLATION --- but don't do
|
|
Packit |
723767 |
that unless you really know what you are doing. Also observe that
|
|
Packit |
723767 |
changes in the global hash will not modify existing collations in
|
|
Packit |
723767 |
existing database handles: it will only affect new *requests* for
|
|
Packit |
723767 |
collations. In other words, if you want to change the behaviour of a
|
|
Packit |
723767 |
collation within an existing $dbh, you need to call the
|
|
Packit |
723767 |
"create_collation" method directly.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
FULLTEXT SEARCH
|
|
Packit |
723767 |
The FTS3 extension module within SQLite allows users to create special
|
|
Packit |
723767 |
tables with a built-in full-text index (hereafter "FTS3 tables"). The
|
|
Packit |
723767 |
full-text index allows the user to efficiently query the database for
|
|
Packit |
723767 |
all rows that contain one or more instances of a specified word
|
|
Packit |
723767 |
(hereafter a "token"), even if the table contains many large documents.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Short introduction to FTS3
|
|
Packit |
723767 |
The detailed documentation for FTS3 can be found at
|
|
Packit |
723767 |
<http://www.sqlite.org/fts3.html>. Here is a very short example :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
$dbh->do(<<"") or die DBI::errstr;
|
|
Packit |
723767 |
CREATE VIRTUAL TABLE fts_example USING fts3(content)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $sth = $dbh->prepare("INSERT INTO fts_example(content) VALUES (?))");
|
|
Packit |
723767 |
$sth->execute($_) foreach @docs_to_insert;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $results = $dbh->selectall_arrayref(<<"");
|
|
Packit |
723767 |
SELECT docid, snippet(content) FROM fts_example WHERE content MATCH 'foo'
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The key points in this example are :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* The syntax for creating FTS3 tables is
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE VIRTUAL TABLE <table_name> USING fts3(<columns>)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where "<columns>" is a list of column names. Columns may be typed,
|
|
Packit |
723767 |
but the type information is ignored. If no columns are specified,
|
|
Packit |
723767 |
the default is a single column named "content". In addition, FTS3
|
|
Packit |
723767 |
tables have an implicit column called "docid" (or also "rowid") for
|
|
Packit |
723767 |
numbering the stored documents.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* Statements for inserting, updating or deleting records use the same
|
|
Packit |
723767 |
syntax as for regular SQLite tables.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* Full-text searches are specified with the "MATCH" operator, and an
|
|
Packit |
723767 |
operand which may be a single word, a word prefix ending with '*', a
|
|
Packit |
723767 |
list of words, a "phrase query" in double quotes, or a boolean
|
|
Packit |
723767 |
combination of the above.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* The builtin function "snippet(...)" builds a formatted excerpt of
|
|
Packit |
723767 |
the document text, where the words pertaining to the query are
|
|
Packit |
723767 |
highlighted.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
There are many more details to building and searching FTS3 tables, so we
|
|
Packit |
723767 |
strongly invite you to read the full documentation at at
|
|
Packit |
723767 |
<http://www.sqlite.org/fts3.html>.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Incompatible change : starting from version 1.31, "DBD::SQLite" uses the
|
|
Packit |
723767 |
new, recommended "Enhanced Query Syntax" for binary set operators (AND,
|
|
Packit |
723767 |
OR, NOT, possibly nested with parenthesis). Previous versions of
|
|
Packit |
723767 |
"DBD::SQLite" used the "Standard Query Syntax" (see
|
|
Packit |
723767 |
<http://www.sqlite.org/fts3.html#section_3_2>). Unfortunately this is a
|
|
Packit |
723767 |
compilation switch, so it cannot be tuned at runtime; however, since
|
|
Packit |
723767 |
FTS3 was never advertised in versions prior to 1.31, the change should
|
|
Packit |
723767 |
be invisible to the vast majority of "DBD::SQLite" users. If, however,
|
|
Packit |
723767 |
there are any applications that nevertheless were built using the
|
|
Packit |
723767 |
"Standard Query" syntax, they have to be migrated, because the
|
|
Packit |
723767 |
precedence of the "OR" operator has changed. Conversion from old to new
|
|
Packit |
723767 |
syntax can be automated through DBD::SQLite::FTS3Transitional, published
|
|
Packit |
723767 |
in a separate distribution.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Tokenizers
|
|
Packit |
723767 |
The behaviour of full-text indexes strongly depends on how documents are
|
|
Packit |
723767 |
split into *tokens*; therefore FTS3 table declarations can explicitly
|
|
Packit |
723767 |
specify how to perform tokenization:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE ... USING fts3(<columns>, tokenize=<tokenizer>)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where "<tokenizer>" is a sequence of space-separated words that triggers
|
|
Packit |
723767 |
a specific tokenizer, as explained below.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SQLite builtin tokenizers
|
|
Packit |
723767 |
SQLite comes with three builtin tokenizers :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
simple
|
|
Packit |
723767 |
Under the *simple* tokenizer, a term is a contiguous sequence of
|
|
Packit |
723767 |
eligible characters, where eligible characters are all alphanumeric
|
|
Packit |
723767 |
characters, the "_" character, and all characters with UTF
|
|
Packit |
723767 |
codepoints greater than or equal to 128. All other characters are
|
|
Packit |
723767 |
discarded when splitting a document into terms. They serve only to
|
|
Packit |
723767 |
separate adjacent terms.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
All uppercase characters within the ASCII range (UTF codepoints less
|
|
Packit |
723767 |
than 128), are transformed to their lowercase equivalents as part of
|
|
Packit |
723767 |
the tokenization process. Thus, full-text queries are
|
|
Packit |
723767 |
case-insensitive when using the simple tokenizer.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
porter
|
|
Packit |
723767 |
The *porter* tokenizer uses the same rules to separate the input
|
|
Packit |
723767 |
document into terms, but as well as folding all terms to lower case
|
|
Packit |
723767 |
it uses the Porter Stemming algorithm to reduce related English
|
|
Packit |
723767 |
language words to a common root.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
icu If SQLite is compiled with the SQLITE_ENABLE_ICU pre-processor
|
|
Packit |
723767 |
symbol defined, then there exists a built-in tokenizer named "icu"
|
|
Packit |
723767 |
implemented using the ICU library, and taking an ICU locale
|
|
Packit |
723767 |
identifier as argument (such as "tr_TR" for Turkish as used in
|
|
Packit |
723767 |
Turkey, or "en_AU" for English as used in Australia). For example:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE VIRTUAL TABLE thai_text USING fts3(text, tokenize=icu th_TH)
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The ICU tokenizer implementation is very simple. It splits the input
|
|
Packit |
723767 |
text according to the ICU rules for finding word boundaries and
|
|
Packit |
723767 |
discards any tokens that consist entirely of white-space. This may
|
|
Packit |
723767 |
be suitable for some applications in some locales, but not all. If
|
|
Packit |
723767 |
more complex processing is required, for example to implement
|
|
Packit |
723767 |
stemming or discard punctuation, use the perl tokenizer as explained
|
|
Packit |
723767 |
below.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Perl tokenizers
|
|
Packit |
723767 |
In addition to the builtin SQLite tokenizers, "DBD::SQLite" implements a
|
|
Packit |
723767 |
*perl* tokenizer, that can hook to any tokenizing algorithm written in
|
|
Packit |
723767 |
Perl. This is specified as follows :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE ... USING fts3(<columns>, tokenize=perl '<perl_function>')
|
|
Packit |
723767 |
|
|
Packit |
723767 |
where "<perl_function>" is a fully qualified Perl function name (i.e.
|
|
Packit |
723767 |
prefixed by the name of the package in which that function is declared).
|
|
Packit |
723767 |
So for example if the function is "my_func" in the main program, write
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE ... USING fts3(<columns>, tokenize=perl 'main::my_func')
|
|
Packit |
723767 |
|
|
Packit |
723767 |
That function should return a code reference that takes a string as
|
|
Packit |
723767 |
single argument, and returns an iterator (another function), which
|
|
Packit |
723767 |
returns a tuple "($term, $len, $start, $end, $index)" for each term.
|
|
Packit |
723767 |
Here is a simple example that tokenizes on words according to the
|
|
Packit |
723767 |
current perl locale
|
|
Packit |
723767 |
|
|
Packit |
723767 |
sub locale_tokenizer {
|
|
Packit |
723767 |
return sub {
|
|
Packit |
723767 |
my $string = shift;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use locale;
|
|
Packit |
723767 |
my $regex = qr/\w+/;
|
|
Packit |
723767 |
my $term_index = 0;
|
|
Packit |
723767 |
|
|
Packit |
723767 |
return sub { # closure
|
|
Packit |
723767 |
$string =~ /$regex/g or return; # either match, or no more token
|
|
Packit |
723767 |
my ($start, $end) = ($-[0], $+[0]);
|
|
Packit |
723767 |
my $len = $end-$start;
|
|
Packit |
723767 |
my $term = substr($string, $start, $len);
|
|
Packit |
723767 |
return ($term, $len, $start, $end, $term_index++);
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
};
|
|
Packit |
723767 |
}
|
|
Packit |
723767 |
|
|
Packit |
723767 |
There must be three levels of subs, in a kind of "Russian dolls"
|
|
Packit |
723767 |
structure, because :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* the external, named sub is called whenever accessing a FTS3 table
|
|
Packit |
723767 |
with that tokenizer
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* the inner, anonymous sub is called whenever a new string needs to be
|
|
Packit |
723767 |
tokenized (either for inserting new text into the table, or for
|
|
Packit |
723767 |
analyzing a query).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
* the innermost, anonymous sub is called repeatedly for retrieving all
|
|
Packit |
723767 |
terms within that string.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Instead of writing tokenizers by hand, you can grab one of those already
|
|
Packit |
723767 |
implemented in the Search::Tokenizer module :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
use Search::Tokenizer;
|
|
Packit |
723767 |
$dbh->do(<<"") or die DBI::errstr;
|
|
Packit |
723767 |
CREATE ... USING fts3(<columns>,
|
|
Packit |
723767 |
tokenize=perl 'Search::Tokenizer::unaccent')
|
|
Packit |
723767 |
|
|
Packit |
723767 |
or you can use "new" in Search::Tokenizer to build your own tokenizer.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Incomplete handling of utf8 characters
|
|
Packit |
723767 |
The current FTS3 implementation in SQLite is far from complete with
|
|
Packit |
723767 |
respect to utf8 handling : in particular, variable-length characters are
|
|
Packit |
723767 |
not treated correctly by the builtin functions "offsets()" and
|
|
Packit |
723767 |
"snippet()".
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Database space for FTS3
|
|
Packit |
723767 |
FTS3 stores a complete copy of the indexed documents, together with the
|
|
Packit |
723767 |
fulltext index. On a large collection of documents, this can consume
|
|
Packit |
723767 |
quite a lot of disk space. If copies of documents are also available as
|
|
Packit |
723767 |
external resources (for example files on the filesystem), that space can
|
|
Packit |
723767 |
sometimes be spared --- see the tip in the Cookbook.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
R* TREE SUPPORT
|
|
Packit |
723767 |
The RTREE extension module within SQLite adds support for creating a
|
|
Packit |
723767 |
R-Tree, a special index for range and multidimensional queries. This
|
|
Packit |
723767 |
allows users to create tables that can be loaded with (as an example)
|
|
Packit |
723767 |
geospatial data such as latitude/longitude coordinates for buildings
|
|
Packit |
723767 |
within a city :
|
|
Packit |
723767 |
|
|
Packit |
723767 |
CREATE VIRTUAL TABLE city_buildings USING rtree(
|
|
Packit |
723767 |
id, -- Integer primary key
|
|
Packit |
723767 |
minLong, maxLong, -- Minimum and maximum longitude
|
|
Packit |
723767 |
minLat, maxLat -- Minimum and maximum latitude
|
|
Packit |
723767 |
);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
then query which buildings overlap or are contained within a specified
|
|
Packit |
723767 |
region:
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# IDs that are contained within query coordinates
|
|
Packit |
723767 |
my $contained_sql = <<"";
|
|
Packit |
723767 |
SELECT id FROM try_rtree
|
|
Packit |
723767 |
WHERE minLong >= ? AND maxLong <= ?
|
|
Packit |
723767 |
AND minLat >= ? AND maxLat <= ?
|
|
Packit |
723767 |
|
|
Packit |
723767 |
# ... and those that overlap query coordinates
|
|
Packit |
723767 |
my $overlap_sql = <<"";
|
|
Packit |
723767 |
SELECT id FROM try_rtree
|
|
Packit |
723767 |
WHERE maxLong >= ? AND minLong <= ?
|
|
Packit |
723767 |
AND maxLat >= ? AND minLat <= ?
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $contained = $dbh->selectcol_arrayref($contained_sql,undef,
|
|
Packit |
723767 |
$minLong, $maxLong, $minLat, $maxLat);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
my $overlapping = $dbh->selectcol_arrayref($overlap_sql,undef,
|
|
Packit |
723767 |
$minLong, $maxLong, $minLat, $maxLat);
|
|
Packit |
723767 |
|
|
Packit |
723767 |
For more detail, please see the SQLite R-Tree page
|
|
Packit |
723767 |
(<http://www.sqlite.org/rtree.html>). Note that custom R-Tree queries
|
|
Packit |
723767 |
using callbacks, as mentioned in the prior link, have not been
|
|
Packit |
723767 |
implemented yet.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
TO DO
|
|
Packit |
723767 |
The following items remain to be done.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Leak Detection
|
|
Packit |
723767 |
Implement one or more leak detection tests that only run during
|
|
Packit |
723767 |
AUTOMATED_TESTING and RELEASE_TESTING and validate that none of the C
|
|
Packit |
723767 |
code we work with leaks.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Stream API for Blobs
|
|
Packit |
723767 |
Reading/writing into blobs using "sqlite2_blob_open" /
|
|
Packit |
723767 |
"sqlite2_blob_close".
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Flags for sqlite3_open_v2
|
|
Packit |
723767 |
Support the full API of sqlite3_open_v2 (flags for opening the file).
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Support for custom callbacks for R-Tree queries
|
|
Packit |
723767 |
Custom queries of a R-Tree index using a callback are possible with the
|
|
Packit |
723767 |
SQLite C API (<http://www.sqlite.org/rtree.html>), so one could
|
|
Packit |
723767 |
potentially use a callback that narrowed the result set down based on a
|
|
Packit |
723767 |
specific need, such as querying for overlapping circles.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
SUPPORT
|
|
Packit |
723767 |
Bugs should be reported via the CPAN bug tracker at
|
|
Packit |
723767 |
|
|
Packit |
723767 |
<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=DBD-SQLite>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Note that bugs of bundled sqlite library (i.e. bugs in "sqlite3.[ch]")
|
|
Packit |
723767 |
should be reported to the sqlite developers at sqlite.org via their bug
|
|
Packit |
723767 |
tracker or via their mailing list.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
AUTHORS
|
|
Packit |
723767 |
Matt Sergeant <matt@sergeant.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Francis J. Lacoste <flacoste@logreport.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Wolfgang Sourdeau <wolfgang@logreport.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Adam Kennedy <adamk@cpan.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Max Maischein <corion@cpan.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Laurent Dami <dami@cpan.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Kenichi Ishigaki <ishigaki@cpan.org>
|
|
Packit |
723767 |
|
|
Packit |
723767 |
COPYRIGHT
|
|
Packit |
723767 |
The bundled SQLite code in this distribution is Public Domain.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
DBD::SQLite is copyright 2002 - 2007 Matt Sergeant.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Some parts copyright 2008 Francis J. Lacoste.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Some parts copyright 2008 Wolfgang Sourdeau.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Some parts copyright 2008 - 2012 Adam Kennedy.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Some parts copyright 2009 - 2012 Kenichi Ishigaki.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
Some parts derived from DBD::SQLite::Amalgamation copyright 2008 Audrey
|
|
Packit |
723767 |
Tang.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
This program is free software; you can redistribute it and/or modify it
|
|
Packit |
723767 |
under the same terms as Perl itself.
|
|
Packit |
723767 |
|
|
Packit |
723767 |
The full text of the license can be found in the LICENSE file included
|
|
Packit |
723767 |
with this module.
|