I just fell upon this thread.
In Europe, there are systems for this already. I think mainly some Northern European countries offer these products.
In general, it is NOT allowed to mix city water with rain water systems, no matter how many check valves you install. The way to do it is to have a sensor in your ground storage tank. When the water level gets below a certain level (just before empty), a valve will open for a set time (say one minute) and refill the tank with an amount of water from the city water. If it goes down to the same low level again, the procedure will repeat itself, effectively keeping the tank from running dry.
As the ground tank is unpressurized, and the city water is pressurized, it is technically impossible for the water to flow the wrong way. Worst case scenario, a valve failure would mean that the storage tank would fill with city water. This is just like the cisterne of a toilet - the water cannot run from the cisterne (unpressurized) back into the city water line (pressurized).
See for instance
YouTube - RewatecUKTV's Channel
Rainwater Harvesting / Rainwater Collection / Sewage Treatment : : Rewatec (the UK website from a German company)
It is important to have the tank underground: Keeps it from freezing in the winter, and from rotting in the summer (depending on your local climate)
A Danish company (no English version)
Regnvandstank (4 m³) - køb regnvandstanke til regnvandsopsamling hos Nyrrup Plast - 1 m3 (cubic meter) is almost 300 USG.