A nil src (NULL database value) will result in a "nil" string,
which will never parse correctly in a ParseInt or similar
numeric conversion. The resulting error is confusing. Check
for a nil src prior to converting the value to string
if the resulting string will be parsed after that.
Closes#31274
Change-Id: I90f12cceff00fbbfdd3e343b04fa7e2596390e6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174177
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL add support for converting a returned cursor (presented
to this package as a driver.Rows) and scanning it into a *Rows.
Fixes#28515
Change-Id: Id8191c568dc135af9e5e8555efcd01987708edcb
Reviewed-on: https://go-review.googlesource.com/c/145738
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously the MaxIdleClosed counter was incremented when added
to the free connection list, rather then when it wasn't added
to the free connection list. Flip this logic to correct.
Fixes#27792
Change-Id: I405302c14fb985369dab48fbe845e5651afc4ccf
Reviewed-on: https://go-review.googlesource.com/c/138578
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The previous implementation would return "sql: Rows are closed" for any
context errors, which can be confusing for context timeouts or
cancelations.
Fixes#24431
Change-Id: I884904ec43204c43f4e94e2335b2802aab77a888
Reviewed-on: https://go-review.googlesource.com/104276
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When 'convertAssign' gives an error, instead of giving just the index of
the failing column -- which is not always helpful, especially when there
are lots of columns in the query -- utilize 'rs.rowsi.Columns()' to
extract the underlying column name and include that in the error string:
sql: Scan error on column index 0, name "some_column": ...
Fixes#23362
Change-Id: I0fe71ff3c25f4c0dd9fc6aa2c2da2360dd93e3e0
Reviewed-on: https://go-review.googlesource.com/86537
Reviewed-by: Harald Nordgren <haraldnordgren@gmail.com>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously we allowed drivers to modify the row buffer used to scan
values when closing Rows. This is no longer acceptable and can lead
to data races.
Fixes#23519
Change-Id: I91820a6266ffe52f95f40bb47307d375727715af
Reviewed-on: https://go-review.googlesource.com/89936
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
While driver.Connector was previously added to allow non-string
connection arguments and access to the context, most users of
the sql package will continue to rely on a string DSN.
Allow drivers to implement a string DSN to Connector interface
that both allows a single parsing of the string DSN and uses
the Connector interface which passes available context to
the driver dialer.
Fixes#22713
Change-Id: Ia0b862262f4c4670effe2538d0d6d43733fea18d
Reviewed-on: https://go-review.googlesource.com/77550
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Russ pointed out in a previous CL golang.org/cl/65731 that not only
was the locking incomplete, previous changes did not correctly
lock driver calls in other sections. After inspecting
driverConn, driverStmt, driverResult, Tx, and Rows structs
where driver interfaces are stored, I discovered a few more places
that failed to lock driver calls. The largest of these
was the parameter type converter "driverArgs".
driverArgs was typically called right before another call to the
driver in a locked region, so I made the entire driverArgs expect
a locked driver mutex and combined the region. This should not
be a problem because the connection is pulled out of the connection
pool either way so there shouldn't be contention.
Fixes#21117
Change-Id: I88d46f74dca25fb11a30f0bf8e79785a73133d23
Reviewed-on: https://go-review.googlesource.com/71433
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes#22049Fixes#20807
Change-Id: I2b5df6d954a38d0ad93bf1922ec16e74c827274c
Reviewed-on: https://go-review.googlesource.com/73033
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit 2620ac3aea.
Reason for revert: broke all the builds.
Change-Id: I26fc09a13f5f80fa708de66c843442ff9d934694
Reviewed-on: https://go-review.googlesource.com/73050
Reviewed-by: Russ Cox <rsc@golang.org>
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes#22049Fixes#20807
Change-Id: Idffa1a7ca9ccfe633257c4a3ae299b864f46c5b6
Reviewed-on: https://go-review.googlesource.com/67630
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Drivers shouldn't need to implement both Queryer and QueryerContext,
they should just implement QueryerContext. Same with Execer and
ExecerContext. This CL tests for QueryContext and ExecerContext
first so drivers do not need to implement Queryer and Execer
with an empty definition.
Fixes#21663
Change-Id: Ifbaa71da669f4bc60f8da8c41a04a4afed699a9f
Reviewed-on: https://go-review.googlesource.com/65733
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit 897080d5cb.
Reason for revert: Fails to fix all the locking issues.
Updates #21117
Change-Id: I6fc9cb7897244d6e1af78c089a2bf383258ec049
Reviewed-on: https://go-review.googlesource.com/71450
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Database drivers should be called from a single goroutine to ease
driver's design. If a driver chooses to handle context
cancels internally it may do so.
The sql package violated this agreement when calling Next or
NextResultSet. It was possible for a concurrent rollback
triggered from a context cancel to call a Tx.Rollback (which
takes a driver connection lock) while a Rows.Next is in progress
(which does not tack the driver connection lock).
The current internal design of the sql package is each call takes
roughly two locks: a closemu lock which prevents an disposing of
internal resources (assigning nil or removing from lists)
and a driver connection lock that prevents calling driver code from
multiple goroutines.
Fixes#21117
Change-Id: Ie340dc752a503089c27f57ffd43e191534829360
Reviewed-on: https://go-review.googlesource.com/65731
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Before this change the ct == 0 check could never be true. Moreover the
values were not properly indirected.
Change-Id: Ice47e36e3492babc4b47d2f9099e8772be231c96
Reviewed-on: https://go-review.googlesource.com/68130
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The current Open method limits the ability for driver maintainers
to expose options for their drivers by forcing all the configuration
to pass through the DSN in order to create a *DB.
This CL allows driver maintainers to write their own initialization
functions that return a *DB making configuration of the underlying
drivers easier.
Fixes#20268
Change-Id: Ib10b794f36a201bbb92c23999c8351815d38eedb
Reviewed-on: https://go-review.googlesource.com/53430
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, the check for `ctx.Done() == context.Background().Done()`
comes before the check to see if we are ignoring any options. That
check should be done earlier, so that the options are not silently
ignored.
Fixes#21350
Change-Id: I3704e4209854c7d99f3f92498bae831cabc7e419
Reviewed-on: https://go-review.googlesource.com/53970
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ensure a Stmt prepared on a Conn executes on the same driver.Conn.
This also removes another instance of duplicated prepare logic
as a side effect.
Fixes#20647
Change-Id: Ia00a19e4dd15e19e4d754105babdff5dc127728f
Reviewed-on: https://go-review.googlesource.com/45391
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rather then write to the same variable per fakeConn, write to either
fakeConn or rowsCursor.
Fixes#20646
Change-Id: Ifc79f989bd1606b8e3ebecb1e7844cce3ad06e17
Reviewed-on: https://go-review.googlesource.com/45393
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In addition to adding a guard to the Rows close, add a var
in the fakeConn that gets read and written to on each
operation, simulating writing or reading from the server.
TestConcurrency/TxStmt* tests have been commented out
as they now fail after checking for races on the fakeConn.
See issue #20646 for more information.
Fixes#20622
Change-Id: I80b36ea33d776e5b4968be1683ff8c61728ee1ea
Reviewed-on: https://go-review.googlesource.com/45275
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The Tx methods Query and Exec uses context.Background()
even Tx was created by context.
This patch enables using Tx.ctx in all Tx methods
which do not has context arg.
Backward compatibility:
- If Tx has created without context, nothing changes.
- If Tx has created with context and non-context method is called:
- If context is expired, the execution fails,
but it can fail on Commit or Rollback as well,
so in terms of whole transaction - nothing changes.
- If context is not expired, nothing changes too.
Fixes#20098
Change-Id: I9570a2deaace5875bb4c5dcf7b3a084a6bcd0d00
Reviewed-on: https://go-review.googlesource.com/44956
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Close any Rows queried within a Tx when the Tx is closed. This prevents
the Tx from blocking on rollback if a Rows query has not been closed yet.
Fixes#20575
Change-Id: I4efe9c4150e951d8a0f1c40d9d5e325964fdd608
Reviewed-on: https://go-review.googlesource.com/44812
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Query and Exec functions on DB first attempt to get a cached
connection before requesting the connection pool to ignore
the cache and get a new connection. This change aligns Stmt to
that behavior as well.
Fixes#20433
Change-Id: Idda5f61927289d7ad0882effa3a50ffc9efd88e6
Reviewed-on: https://go-review.googlesource.com/43790
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously all arguments were passed through driver.IsValid.
This checked arguments against a few fundamental go types and
prevented others from being passed in as arguments.
The new interface driver.NamedValueChecker may be implemented
by both driver.Stmt and driver.Conn. This allows
this new interface to completely supersede the
driver.ColumnConverter interface as it can be used for
checking arguments known to a prepared statement and
arbitrary query arguments. The NamedValueChecker may be
skipped with driver.ErrSkip after all special cases are
exhausted to use the default argument converter.
In addition if driver.ErrRemoveArgument is returned
the argument will not be passed to the query at all,
useful for passing in driver specific per-query options.
Add a canonical Out argument wrapper to be passed
to OUTPUT parameters. This will unify checks that need to
be written in the NameValueChecker.
The statement number check is also moved to the argument
converter so the NamedValueChecker may remove arguments
passed to the query.
Fixes#13567Fixes#18079
Updates #18417
Updates #17834
Updates #16235
Updates #13067
Updates #19797
Change-Id: I89088bd9cca4596a48bba37bfd20d987453ef237
Reviewed-on: https://go-review.googlesource.com/38533
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When running a Query on Stmt a dependency is added to the stmt and
rows. To do that it needs a reference to Rows, so the releaseConn
function is defined after the definition. However the
rows.initContextClose was set to run before the releaseConn was
set on rows, setting up a situation where the connection could
be canceled before the releaseConn was set and resulting in
a segfault.
Fixes#20160
Change-Id: I5592e7db2cf653dfc48d42cbc2b03ca20501b1a0
Reviewed-on: https://go-review.googlesource.com/42139
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Databases have the following concepts: Statement, Batch, and Session.
A statement is often a single line like:
SELECT Amount from Account where ID = 50;
A batch is one or more statements submitted together for the query
to process. It may be a DELETE, INSERT, two UPDATES and a SELECT in
a single query text.
A session is usually represented by a single database connection.
This often is an issue when dealing with scopes in databases.
Temporary tables and variables can have batch, session, or global
scope depending on the syntax, database, and use.
Furthermore, some databases (sybase and derivatives in perticular)
that prevent certain statements from being in the same batch
and may necessitate being in the same session.
By allowing users to extract a Conn from the database they can manage
session on their own without hacking around it by making connection
pools of single connections (a real workaround presented in issue).
It is tempting to just use a transaction, but this isn't always
desirable or an option if running an interactive session or
alter script set that itself starts transactions.
Fixes#18081
Change-Id: I9bdf0796632c48d4bcaef3624c629641984ffaf2
Reviewed-on: https://go-review.googlesource.com/40694
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When a Tx starts a query, prevent returning the connection to the pool
until after the query finishes.
Fixes#19058
Change-Id: I2c0480d9cca9eeb173b5b3441a5aeed6f527e0ac
Reviewed-on: https://go-review.googlesource.com/40400
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
User defined numeric types such as "type Int int64" have
been able to be scanned into without a custom scanner by
using the reflect scan code path used to convert between
various numeric types. Add in a path for string types
for symmetry and least surprise.
Fixes#18101
Change-Id: I00553bcf021ffe6d95047eca0067ee94b54ff501
Reviewed-on: https://go-review.googlesource.com/39031
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change was originally written by Marko Tiikkaja <marko@joh.to>.
https://go-review.googlesource.com/#/c/2035/
Previously *Tx.Stmt always prepared a new statement, even if an
existing one was available on the connection the transaction was on.
Now we first see if the statement is already available on the
connection and only prepare if it isn't. Additionally, when we do
need to prepare one, we store it in the parent *Stmt to allow it to be
later reused by other calls to *Tx.Stmt on that statement or just
straight up by *Stmt.Exec et al.
To make sure that the statement doesn't disappear unexpectedly, we
record a dependency from the statement returned by *Tx.Stmt to the
*Stmt it came from and set a new field, parentStmt, to point to the
originating *Stmt. When the transaction's *Stmt is closed, we remove
the dependency. This way the "parent" *Stmt can be closed by the user
without her having to know whether any transactions are still using it
or not.
Fixes#15606
Change-Id: I41b5056847e117ac61130328b0239d1e000a4a08
Reviewed-on: https://go-review.googlesource.com/35476
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
When testing context cancelation behavior do not rely on context
timeouts. Use explicit checks in all such tests. In closeDB
convert the simple check for zero open conns with a wait loop
for zero open conns.
Fixes#19024Fixes#19041
Change-Id: Iecfcc4467e91249fceb21ffd1f7c62c58140d8e9
Reviewed-on: https://go-review.googlesource.com/36902
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Previously if a context was canceled while it was waiting for a
connection request, that connection request would leak.
To prevent this remove the pending connection request if the
context is canceled and ensure no connection has been sent on the channel.
This requires a change to how the connection requests are represented in the DB.
Fixes#18995
Change-Id: I9a274b48b8f4f7ca46cdee166faa38f56d030852
Reviewed-on: https://go-review.googlesource.com/36563
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously it was intended that Rows.Scan would return
an error and Rows.Err would return nil. This was problematic
because drivers could not differentiate between a normal
Rows.Close or a context cancel close.
The alternative is to require drivers to return a Scan to return
an error if the driver is closed while there are still rows to be read.
This is currently not how several drivers currently work and may be
difficult to detect when there are additional rows.
At the same time guard the the Rows.lasterr and prevent a close
while a Rows operation is active.
For the drivers that do not have Context methods, do not check for
context cancelation after the operation, but before for any operation
that may modify the database state.
Fixes#18961
Change-Id: I49a25318ecd9f97a35d5b50540ecd850c01cfa5e
Reviewed-on: https://go-review.googlesource.com/36485
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously the following could happen, though in practice it would
be rare.
Goroutine 1:
(*Tx).QueryContext begins a query, passing in userContext
Goroutine 2:
(*Tx).awaitDone starts to wait on the context derived from the passed in context
Goroutine 1:
(*Tx).grabConn returns a valid (*driverConn)
The (*driverConn) passes to (*DB).queryConn
Goroutine 3:
userContext is canceled
Goroutine 2:
(*Tx).awaitDone unblocks and calls (*Tx).rollback
(*driverConn).finalClose obtains dc.Mutex
(*driverConn).finalClose sets dc.ci = nil
Goroutine 1:
(*DB).queryConn obtains dc.Mutex in withLock
ctxDriverPrepare accepts dc.ci which is now nil
ctxCriverPrepare panics on the nil ci
The fix for this is to guard the Tx methods with a RWLock
holding it exclusivly when closing the Tx and holding a read lock
when executing a query.
Fixes#18719
Change-Id: I37aa02c37083c9793dabd28f7f934a1c5cbc05ea
Reviewed-on: https://go-review.googlesource.com/35550
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Slower builders were failing TestQueryContext because the cancel
and return to conn pool happens async. TestQueryContext already
uses a wait method for this reason. Use the same method for
other context tests.
Fixes#18759
Change-Id: I84cce697392b867e4ebdfadd38027a06ca14655f
Reviewed-on: https://go-review.googlesource.com/35750
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Clean up the phrasing a little bit, make the comment fit in 80
characters, and fix the spelling of "guard."
Change-Id: I688a3e760b8d67ea83830635f64dff04dd9a5911
Reviewed-on: https://go-review.googlesource.com/34792
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously Tx.done was being set in close, but in a Tx
rollback and Commit are the real closing methods,
and Tx.close is just a helper common to both. Prior to this
change a multiple rollback statements could be called, one
would enter close and begin closing it while the other was
still in rollback breaking it. Fix that by setting done
in rollback and Commit, not in Tx.close.
Fixes#18429
Change-Id: Ie274f60c2aa6a4a5aa38e55109c05ea9d4fe0223
Reviewed-on: https://go-review.googlesource.com/34716
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Drivers which previously supported tip will need to update to this
revision before release.
Fixes#18284
Change-Id: I70b8e7afff1558a8b5348885ce9f50e067c72ee9
Reviewed-on: https://go-review.googlesource.com/34330
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Require parameter names to not begin with a symbol.
Change-Id: I5dfe9d4e181f0daf71dad2f395aca41c68678cbe
Reviewed-on: https://go-review.googlesource.com/33493
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rather then using a sleep in the fake DB, go to a channel
select and wait for the context to be done.
Fixes#18115
Change-Id: I6bc3a29db58c568d0a7ea06c2a354c18c9e798b2
Reviewed-on: https://go-review.googlesource.com/33712
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
