If you can find one, a lower mileage '95 Ej22 is the way to go, engine wise. Non-interference, so no major damage if a timing belt lets go. Also stupid easy to work on with basic tools... but as GD said, it will be under-powered for what you are wanting to do.
beyond that, best I can offer is to make sure ALL maintenance is up to date. Fluids, brakes, cooling system, and so on.
I get the sentimentality of keeping the car, but in all honesty, I would fix it up for a daily driver, and get something else more suited for the offroad adventures.
Can you go "offroad" with it? yes.
Are they fun to do a little mild offroading with? most definitely, but as mentioned, will not be nearly as capable as the vehicles mentioned by GD.