Building is different than doing a compose up
. If you're making changes to the dockerfile and want to build your changes, then DOCKER_BUILDKIT=1 docker build -t custom/lemmy: .
should be the correct method.
Then you would change the docker-compose file to reference your newly built image instead of using the docker hub one. See this document.