This would work but assumes the primary use of the machine is Windows and derates your performance under Linux significantly due to USB speeds. Even if you're storing your data on the Windows HDD, NTFS drivers are dog slow compared to EXT4 and other *nix filesystems.
Also some BIOSes are a pain to get to boot off removable drives reliably so it really depends on what your machine is.
I've used Linux as a primary dev system for well over a decade now, and with the current state of Windows I'd really recommend just taking the leap, keep your Windows box if you need Windows software and build a dedicated Linux workstation.
This is a pretty good idea, my wife dual boots and I'll suggest it to her as Windows keeps trashing the EFI partition.