Skip to content

API for inserting multiple rows fast #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
kkaefer opened this issue Feb 25, 2011 · 7 comments
Closed

API for inserting multiple rows fast #2

kkaefer opened this issue Feb 25, 2011 · 7 comments
Labels
feature request

Comments

@kkaefer
Copy link
Contributor

kkaefer commented Feb 25, 2011

No description provided.

@fictorial
Copy link

?

@kkaefer
Copy link
Contributor Author

kkaefer commented Jun 2, 2011

This is a feature request. The feature would provide an API for batching large amounts of rows from JS to C++ land. The idea is to improve performance. However, we haven't found any performance bottlenecks in this area for our use case, so it's not implemented yet.

@fictorial
Copy link

a-ha!

@klaufir
Copy link

klaufir commented Jan 27, 2014

This is a must have. Well established DB-APIs all have some way to batch insert values. See Python's executemany. http://www.python.org/dev/peps/pep-0249/#id18 .

springmeyer pushed a commit that referenced this issue May 4, 2018
@anisha-rohra @mhdawson
c++11 requirement for xcode (#2)
6021084
@markmitton markmitton mentioned this issue Jul 18, 2018
Closed
@RobertSasak
Copy link

I recommend adding following. It has speed up inserting in order of magnitude. Source: http://blog.quibb.org/2010/08/fast-bulk-inserts-into-sqlite/

db.run('PRAGMA synchronous=OFF')
db.run('PRAGMA count_changes=OFF')
db.run('PRAGMA journal_mode=MEMORY')
db.run('PRAGMA temp_store=MEMORY')

@cekvenich
Copy link

What RobertSasak said plus begin/commit. I get ~ 30K rows/sec.

paulfitz pushed a commit to paulfitz/node-sqlite3 that referenced this issue Apr 29, 2021
@jvorob
Fixed testing on linux, readme, electron test (TryGhost#2)
de2475e 
* Fixed testing on linux, readme, electron test

marshal.node failed to link on linux for marshal-test.js
specified library in test/cpp/binding.gyp

Electron test only worked if repo was checked out as exactly node-sqlite3
also, module was sometimes cached, breaking the test (needed to delete it to force reload)
@shinyeasy shinyeasy mentioned this issue Jul 7, 2022
Open
@daniellockyer
Copy link
Member

I like the PRAGMA suggestions in this thread, so I don't think we explicitly need a specific API right now 🙂

@daniellockyer daniellockyer closed this as not planned Won't fix, can't repro, duplicate, stale Sep 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@kkaefer @fictorial @RobertSasak @daniellockyer @klaufir @cekvenich