Curious where you go with this project. I was down this same road about 7-8 months ago. I havnt abandoned the project, I just figured out a better way to do it and havnt had the time to start over yet.
The tails of that project are HERE . In my setup, the next step before I canceled the current design was to add a condenser stage before the exhaust air coil to dump some of the extra energy to the outside air, so it could take more complete use of the exhausting air. You could theoretically use your geo for this, giving you a BPHE before the outgoing air coil. The geo would take the brunt of the energy, leaving the air coil to come to nearly the same temp as the exiting air. The only real difference between our projects is mine is intentionally for summer use, yours is for winter use. Having the extra exchanger would offer you a net gain to your interior space rather than a break even of "this for that". At least in a more dramatic fashion since your making the unit more efficient at its job by over-sizing one side of the equation. As far as compressor size, I dont think you are too far off your mark. On the cooling side of things, I was using a 9k portable in a "100%" fresh air configuration with no reclaim and was quite happy with the inside conditions. Granted were talking about much higher deltas on your cold end than I on the hot end. Variable speed fans are your friend, as that is what really determines your temperature extremes, not the compressor.