107
submitted 11 months ago by otl@lemmy.sdf.org to c/linux@lemmy.ml

lemmyverse: search lemmy communities from the command-line. Thanks to the data HTTP API from lemmyverse.net! This is not really as polished as I like but, hey, in the interest of having a lively Lemmy I thought I'd share anyway :)

Usage

lemmyverse searches community names and descriptions using a regular expression:

lemmyverse pattern

Find communities about motorcycles:

$ lemmyverse motorcycle
120024@lemmy.world      All Things motorcycles
20hirnzelle@feddit.ch   All Things motorcycles
7810322@lemmy.world     All Things motorcycles
bmwmotorrad@lemmy.world Community for BMW motorcycles. A place to share
bootstrappable@slrpnk.net       A community to discuss all things BMW cars & motorcycles.\nFeel free to show off your new vehicle/parts
buell@lemmy.world       A discussion area for Buell motorcycles.
motorcycle_logistics@lemmy.world        A community for pictures and videos of people using motorcycles to transport things in a creative manner.\n\nThis includes
motorcycles@feddit.de   This community is for all things motorcycle related. At a later point and with enough traction gained
...

Find communities for the Plan 9 operating system:

$ lemmyverse '(plan9)|(Plan 9)'
plan9@lemmy.sdf.org     Discussions on the Plan9 operating system.

Why?

I run relatively slow hardware and I'm travelling in Bali, Indonesia at the moment. Loading lemmyverse.net in a web browser takes ages and gets the laptop fans spinning (it's hot here!). So I had some fun creating a tiny command-line program to find Lemmy communities using classic UNIX tools awk(1), tr(1), grep(1) etc.

More info

See the man page:

LEMMYVERSE(1)               General Commands Manual              LEMMYVERSE(1)

NAME
     lemmyverse - find lemmy communities

SYNOPSIS
     lemmyverse pattern

DESCRIPTION
     lemmyverse finds Lemmy communities indexed by lemmyverse.net using the
     given regular expression as interpreted by grep(1).  Both the names and
     descriptions of the communities are searched.

     On first run, a local community database must be generated.  The full
     community index is downloaded from https://lemmyverse.net using curl(1),
     transformed, then stored in the user cache directory.  To regenerate the
     database, remove the file and run lemmyverse again.

FILES
     communities
             Community database from lemmyverse.net.

ENVIRONMENT
     lemmyverse uses the following environment variables:

     XDG_CACHE_DIR
             The directory to store the community database.  If unset,
             $HOME/.cache/lemmyverse is used.

EXAMPLES
     Find communities for the Plan 9 operating system:
           lemmyverse '(plan9)|(Plan 9)'

     List all communities from the instance lemmy.sdf.org:
           lemmyverse '@lemmy.sdf.org'

EXIT STATUS
     The lemmyverse utility exits 0 on success, and >0 if an error occurs.

SEE ALSO
     grep(1), curl(1), https://lemmyverse.net
top 11 comments
sorted by: hot top controversial new old
[-] zazaserty@discuss.tchncs.de 8 points 11 months ago

Thats a great command line utility!

[-] freamon@endlesstalk.org 8 points 11 months ago

Lemmyverse is great - I use it to generate the lists at Trending Communities.

Re: the "thousands of spam/bot communities at lemmy.world?" question in your source, yeah there's about 6k. A user tried to create about 2000, all called "serverhold" if you want a way to exclude them, got banned, and in retaliation created about 4000 with that junk string.
The admin of lemmyworld said it'll take time to remove them, but I don't know why (there seems to be the functionality in the API)

[-] otl@lemmy.sdf.org 3 points 11 months ago

Oh that's interesting - nice spotting.

The admin of lemmyworld said it’ll take time to remove them, but I don’t know why (there seems to be the functionality in the API)

Indeed. Looks like it's just a matter of a big for loop over each community ID, POST /community/delete with body community_id=1234. Maybe I'll try to see if it's possible to get some action on removing them by sending over some Python script or something that does the job. Quick count shows almost 14%(!) of all communities indexed by lemmyverse are those junk communities:

% lemmyverse . | wc -l
  30376
% lemmyverse enoweiooe | wc -l
   4206
[-] mrmanager 2 points 11 months ago

You can probably also delete them directly in the instance database with simple sql. Unless there are dependencies that will cascade delete things that are not wanted. :)

[-] otl@lemmy.sdf.org 1 points 11 months ago

Yeah that's another way. I'm pretty terrible with relational databases, though! I've sent over a Python script which does the job via the HTTP API. Hopefully the admins of lemmy.world will see the post: https://lemmy.world/post/1720870

[-] otl@lemmy.sdf.org 1 points 11 months ago

Follow up: I wrote a script to delete all the communities made by that user. https://lemmy.world/post/1720870 Feel free to upvote for visibility ;)

[-] freamon@endlesstalk.org 2 points 11 months ago

Will do. Hope they use it, having all that junk in the JSON file is no use to anyone.

[-] garam@lemmy.my.id 4 points 11 months ago

Thanks for sharing, and hello from Regional Indonesian/Bali instance!

[-] otl@lemmy.sdf.org 2 points 11 months ago

Selamat malam dan terimah kasih :) Nenak saya berasal dari Java, tetapi bahasa indonesia saya tidak bagus :( Saya belajar !

[-] garam@lemmy.my.id 3 points 11 months ago

Selamat malam!

Belajar saja, semua belajar. Inggris juga bukan bahasa Ibu saya. 😂

Saya sendiri tak bisa bahasa Ibu saya, hanya. Bisa bahasa Indonesia 😂🤪

Learn Javanese, it will be fun,

[-] mrmanager 4 points 11 months ago

Very nice tool, will use this for sure. I prefer command line also, so much faster.

this post was submitted on 18 Jul 2023
107 points (100.0% liked)

Linux

45530 readers
1468 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS