In yoga triangle pose the feet are separated about a leg's length with one foot turned in slightly and the other foot turned out 90 degrees. Both feet are flat on the floor and both knees are straight. The body is tilted towards the "turned-out" foot with the ribcage over the leg. Both the ribcage and pelvis "face the front".
When entering the pose the first thing I have my students do is set up their feet with one foot turned in and the other foot turned out. From there the next step is to tilt the pelvis towards the turned out foot. With the right foot turned out 90 degrees the pelvis is tilted to the right so that the right hip is lower than the left hip. The ribcage naturally follows the tilt of the pelvis and ends up leaning over the leg.
While it can be easy to lean forwards, in this yoga pose I'd suggest that you can get used to this positioning by using your breath.
In ashtanga yoga the usual instruction is to grab the big toe with the first two fingers and thumb of the lower hand. Holding onto the big toe with the bottom hand you can then focus on pulling the ribcage upwards, away from the leg. At the same time you reach up with the top arm and look up towards the top thumb.
An alternative for triangle pose is to focus on gradually letting the ribcage sink down without worrying about grabbing the big toe.
To begin with start with the hand on the thigh or shin with the elbow straight.
Slowly bend the elbow to let your ribcage sink down. Then brace your leg, and lift your hand off of your shin. Do this while inhaling and to rest, put the hand back on your shin while exhaling. As you get used to supporting your upper body using only your legs, you can add weight to the pose by reaching first one hand and then both hands to the side.
Initially you might choose to lift the hand while inhaling and support the ribcage with the bottom hand while exhaling. But then you can simply hold yoga triangle pose with both hands reaching past the head and focus on breathing slowly and smoothly.
Rather than keeping the ribcage lifted let it sink gradually towards your leg.
While holding this version of triangle pose
Note that as you sink deeper, instead of having to place your hand on your shin you can place it on the floor behind your leg.
You may also be interested in Twisting Yoga Triangle Pose (Parivrtta Trikonasana)
For some simple stabilizing techniques you can use in this and other poses check out the ebook: Yoga for Beginners 1.
If you prefer learning via videos, the Extreme Stability for Yoga Video provides a different way of stabilizing the legs, working from ground up.