Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mamot.fr is one of the many independent Mastodon servers you can use to participate in the fediverse.
Mamot.fr est un serveur Mastodon francophone, géré par La Quadrature du Net.

Administered by:

Server stats:

3.3K
active users

mamot.fr: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.4

Antonin Delpeuch

Optimizing is fun! Today I took it for a ride on the database (all french companies). The CSV file weighs 7.8GB uncompressed. It took a while to import it, but… tada!

The OpenRefine UI on a project with 81,692,446 rows, showing the first few rows in it… and not choking!
Apr 16, 2023, 11:26 AM·Public·Web
6boosts·17favorites
Quiet public
Antonin Delpeuch

This is using the new architecture for the upcoming 4.0 version. There are still quite a few cases in which the tool will definitely choke on a dataset that big, but I am working on those one step at a time.

Quiet public
Antonin Delpeuch

I have to say 81M rows is beyond the range of what I expect people to use the tool for, but the many performance issues that show up with such a dataset are often worth addressing anyway, because they will still make a difference on smaller datasets.

ExploreLive feeds
About