[-] liori@lemm.ee 1 points 4 months ago

Personally I think child processes are the right approach for this. Launch a new process* for each query and it can (if you choose to go that route) dynamically load in compiled code. Exit when you’re done, and the dynamically loaded code is gone. A side benefit of that is memory leaks are contained, since all memory you allocate is about to be removed anyway.

I'd probably be fine with hundreds or thousands of these hanging in memory. I suspect the generated code for a single query would be in hundreds of kilobytes, maybe a megabyte. But yeah, this is one of those technical details I'd worry about.

Honestly, I wonder if you could just use an actual HTTP server for this? They can handle hundreds or even thousands of simultaneous requests. They can handle requests that complete in a fraction of a millisecond or ones that run for several hours. And they have good tools to catch/deal with code that segfaults, hits an endless loop, attempts to allocate terabytes of swap, etc. HTTP also has wonderful tools to load balance across multiple servers if you do need to scale to massive numbers of requests.

Not sure how a HTTP server would solve the CPU bottleneck of scanning terabytes of data per query?

[-] liori@lemm.ee 2 points 4 months ago

I somehow didn't think a regular JIT solution might be applicable here, but it is. Thank you! There seems to be a number of projects doing JIT for C++, will look at them.

[-] liori@lemm.ee 3 points 7 months ago

Plenty of them on various sites, like this one I found yesterday.

[-] liori@lemm.ee 2 points 10 months ago* (last edited 10 months ago)

Try dmraid, it's been designed to take over various formats of hardware RAID cards.

[-] liori@lemm.ee 2 points 10 months ago

The thread is an attempt to merge a new file system, bcachefs. This is a large change, requiring a lot of review from experienced developers, and getting anyone to do this work turned out to be difficult. Darrick here started talking how, in general, all development of file systems in Linux is troubled by lack of manpower.

[-] liori@lemm.ee 1 points 11 months ago* (last edited 11 months ago)

I do not have notes from that time anymore, sorry. I do recall though that after following a chain of citations I ended up at the paper in the center of this controversy. Nobody sane would cite in now except to point out its flaws, but if there's a modern paper that cites a 10 year old paper that cites a 30 year old paper that cites it—people usually won't notice.

[-] liori@lemm.ee 3 points 11 months ago

Good question! I quickly found this table, though this is yearly statistics only: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3510019201

[-] liori@lemm.ee 1 points 1 year ago

Will they keep the dense email list view as an option? Seeing more than the 14 email messages visible on the screenshot in the post is useful to sort out large folders.

[-] liori@lemm.ee 1 points 1 year ago* (last edited 1 year ago)

I'm surprised federation isn't based on asymmetric cryptography. Let the public/private keys identify instances, as opposed to domains that risk being blocked by governments or bought by malicious third parties if the instance owner forgets to prolong it.

With that, implementing a change in domain names would be simple.

[-] liori@lemm.ee 1 points 1 year ago

Last job, we started writing mixing bits of Kotlin in an otherwise mostly-Java in a monolithic Spring-based service. Good experience.

[-] liori@lemm.ee 2 points 1 year ago* (last edited 1 year ago)

Mosh is not more secure than SSH: it uses SSH to initiate connections, so this is the upper bound.¹ Mosh's UDP protocol has not been given the same security attention as SSH's, though. As such, I'm willing to use it over the open internet for private stuff, but not for business use cases.

¹ I know mosh authors write «In one concrete respect, the Mosh protocol is more secure than SSH's: SSH relies on unauthenticated TCP to carry the contents of the secure stream.», but this refers to the UDP stream after the connection is initiated. If there is a security hole in SSH, it's pretty likely the attacker could take advantage of it during the connection initiation process. Mosh authors do acknowledge this in the subsequent paragraph: «However, in typical usage, Mosh relies on SSH to exchange keys at the beginning of a session, so Mosh will inherit the weaknesses of SSH—at least insofar as they affect the brief SSH session that is used to set up a long-running Mosh session.»

[-] liori@lemm.ee 2 points 1 year ago

Indeed. Most of the web is broken under GDPR's privacy requirements.

view more: ‹ prev next ›

liori

joined 1 year ago