Hello @remcopoelstra,
639k rows is a lot.
Make sure you perform all your changes in a single transaction. That's key for SQLite performance.
If you want to discuss more, please open a GitHub issue, and provide enough information. A small and focused reproducible case would be great. Add detailed profiling information. And your own initial analysis of what could be eventually improved in the library. Thank you!