I did that when my son was 3, maybe 4. Just he and I went out, set up the trailer for the weekend, and launched the boat. That afternoon the worst storm in a bunch of years hit. I think 8 boats sank on Havasu. We big rolling white caps in the river between Needles and Moabi. We did fine, he got a little scared, but handled it good. The only problem putting the boat back on the trailer in the heavy wind storm by myself! I finally was able to get the guy pulling out next to me to drive up my truck while I held the boat in place.
For the rest of the season my son was always "checking for clouds" before we launched the boat. LOL
Waterndog and I have since had "no Moms" trips with my two kids and his daughter. Kids loved that. Two Dads and three kids, all under 6yo.
You should have a great time.
RTJas