hexbear

10254 readers

1 users here now

Now that the old Hexbear fork has been officially abandoned, this community will be used as a space for meta-discussion on the site itself.

founded 3 years ago

MODERATORS

eiknat@hexbear.net

Ryaina@hexbear.net

ThePeoplesGuillotine@hexbear.net

DashEightMate@hexbear.net

HexbearPR@hexbear.net

Gardener@hexbear.net

ella@hexbear.net

External thumbnail URLs not proxied (hexbear.net)

submitted 2 months ago* (last edited 2 months ago) by wheresmysurplusvalue@hexbear.net to c/hexbear@hexbear.net

7 comments fedilink hide all child comments

I noticed while creating a new post that if I provide a Thumbnail URL with an external (not hexbear.net) URL, then the original image gets used when viewing the post. I first noticed it on this post where I added a youtube thumbnail icon (check it in the browser console).

I tried researching if there's a bug for this in upstream Lemmy:

This one seems like it would have fixed it: LemmyNet/lemmy-ui: Update post listing to prefer local image when available

Unfortunately I'm not that familiar with the Lemmy codebase, but how possible would it be to proxy+cache thumbnails? I don't quite mean store it permanently in the database, just proxy the source image and cache it for performance reasons. Otherwise, maybe we should disable the thumbnail URL for now until this support is added upstream?

you are viewing a single comment's thread
view the rest of the comments

[–] PorkrollPosadist@hexbear.net 2 points 2 months ago* (last edited 2 months ago) (3 children)

Unfortunately I'm not that familiar with the Lemmy codebase, but how possible would it be to proxy+cache thumbnails?

As far as I know, there is no infrastructure for this at all. As an alternative, Hexbear prevents embedding remotely-hosted images in comments/posts, but as you have pointed out, this does not seem to be a 100% foolproof measure. There are other Fediverse platforms which do cache all remote media, but this comes at a cost. Matapacos.dog, an instance of about 100 active users, is currently storing 2.3 TB of cached media. (There's a much smaller 48 hour hot cache on the VPS, this is the 1 month (IIRC) secondary cache which lives in an S3-compatible object storage bucket).

[–] PorkrollPosadist@hexbear.net 3 points 2 months ago (1 children)

I'm beginning to think something is preventing this cache from being cleaned periodically though lmao.

[–] PorkrollPosadist@hexbear.net 4 points 2 months ago

mastodon@matapacos:~/live$ bin/tootctl media remove --dry-run --days 30
2625433/2625777 |============================================== |  ETA: ??:??:??
Removed 2625766 media attachments (approx. 1.7 TB) (DRY RUN)

:yea:

[–] Speaker@hexbear.net 2 points 2 months ago

What's your object store? I have a couple projects that would be easier with something S3-compatible, but I would love to not give Jeffrey Kisses any money for the privilege.

[–] wheresmysurplusvalue@hexbear.net 1 points 2 months ago (1 children)

Interesting stats! Yeah, it sounds like storing everything remote that ever touched hexbear is going to balloon storage costs. I guess if there's no infrastructure for it then it just waits for something to be offered for it by the Lemmy devs. Maybe thumbnails are less important and could just be proxied without long term storage, and if the source image dies, well it's just a thumbnail.

[–] PorkrollPosadist@hexbear.net 2 points 2 months ago* (last edited 2 months ago)

IMO the cost is immaterial. It is a design decision. Less caching makes the start-up cost of instances lower. More caching makes the instances safer to use, but increases the requirements of running one. Either choice could be correct depending on the context.

Yeah, it sounds like storing everything remote that ever touched hexbear is going to balloon storage costs.

'Balloon' in this case is still manageable. In my case, this storage still costs less than $20 a month, (the minimum is about $8/mo for 1TB or less). If you're paying Amazon prices you're fucked though. lord-bezos-amused