If the filter is cheap and easy to replace I don't see it being a big deal. I'd just want to make sure it filters quite well. You really don't want to have to take apart the heat exchanger and clean it out if you don't have to.
But, nobody has ever put a filter on a dryer heat exchanger. So, we don't really know how bad it is for sure.