this post was submitted on 19 May 2025
51 points (98.1% liked)

Programming

21284 readers
1 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] Boomkop3@reddthat.com 8 points 1 month ago (2 children)

I have a bit of a radical approach preventing these issues: don't allow unicode unless explicitly indicated or in contexts where it's required or harmless

[–] floofloof@lemmy.ca 12 points 1 month ago* (last edited 1 month ago) (1 children)

That sounds good, but when you start thinking about how to implement this practically, it seems like it would either be unfeasible or would fail to really address the problem.

Maybe the rule would be: Unicode is allowed only in resource files. It would make code comments awkward for many non-English-speaking programmers. But suppose you did it, then since URLs can include Unicode, it would become normal to put URLs in resource files. If the VCS flagged up Unicode commits in source code, it would have to give resource files a pass. So in any case where you're not hardcoding a URL it wouldn't flag up Unicode URL abuses like the one illustrated here. You wouldn't really have fixed the problem, just hidden it in a different way. You'd still need to flag up ambiguous Unicode characters in resource files.

[–] Boomkop3@reddthat.com 1 points 1 month ago

A linter can get quite a bit more in depth than a file-to-file approach. At the very least you can use an ambigious character block list for function names

[–] Bogasse@lemmy.ml 6 points 1 month ago

That's exactly the solution from the article 👌

In the curl git repository most files and most content are plain old ASCII so we can “easily” whitelist a small set of UTF-8 sequences and some specific files, the rest of the files are simply not allowed to use UTF-8 at all as they will then fail the CI job and turn up red.