The hamstrings are a set of muscles at the back of the thigh that work on both the back of the hip joint and the back of the knee. Other muscles that work on the back of the hip include the gluteus maximus and the adductor magnus long head. Those muscles are included in the following discussion.
Anatomically it makes sense to differentiate between the muscles that work on both the back of the hip and the knee (multi joint or poly-articulate muscles) versus those that work just on the back of the knee or just on the back of the hip (single joint or mono-articulate muscles).
The gluteus maximus is actually relatively complex in terms of anatomy in that it could be divided into two (or perhaps even three) different muscles.
It is this latter group of fibers that can be used to bend the hip backwards.
The standing hip thrust is one way of activating the gluteus maximus muscle.
Standing with feet about hip width apart, activate the glutes and push the hips forwards.
You may find that it feels better to activate the glutes first and then push the hips forwards. However, also try pushing the hips forwards and activating the glutes as you do so.
When activating your glutes for this exercise, prevent your thigh bones from rotating externally.
One method for activating the glutes is to imagine pinching a penny between your buttocks. However, this causes the thighs to externally rotate.
The action you are trying to do in this pose is using the glutes to push your hips forwards.
This glute squeezing action can be done without externally rotating the thighs.
The adductor-magnus is another large leg and hip muscle that can be divided into parts. The portion that is relevant to this article is sometimes called the adductor-magnus-long-head. It attaches from the sitting bone to the bottom of the thigh bone, just above the inside of the knee. The main difference between it and the hamstring muscles is that it doesn't cross the knee joint.
Like the gluteus maximus this muscle too can be used to bend the hip backwards. Where the gluteus maximus is quite bulky the adductor-magnus-long-head is quite thin. One way that I teach students to use this muscle is in cat pose with one leg lifted with the knee straight.
So that the knee of the lifted leg easily points downwards (it's easy to allow the lift the leg in such a way that the knee points slightly outwards) feel like you are pulling upwards on the inside edge of the thigh, just above the knee. Pull upwards on the inside edge of the thigh and allow the weight of the bulk of the thigh sink down.
The adductor-magnus-long-pulls upwards on the inner thigh while the weight of the thigh pulls downwards and you may find that the relative effort required to keep the leg is quite small.
See if you can use this same set of actions to lift the leg higher.
The hamstring muscles consist of four muscles, the semitendinosus, semimembranosus, biceps femoris long head and biceps femoris short head.
Some anatomists don't consider the biceps-femoris-short-head to be a "hamstring muscle" because it only crosses the back of the knee, and not the hip. I don't think it matters because I'm more concerned with the muscle itself versus the names of the muscle or what group they belong too.
Obviously being consistent in the use of names of muscles is important so that we know which muscles we are talking about, and so to avoid confusion I'll refer to the muscles by their individual name where it is important to differentiate them.
The biceps femoris short head attaches to the back of the femur. It crosses the back of the knee to attach to the fibula. It and the popliteus are the only two muscles that cross the back of the knee and act only on the knee. (The gastrocnemius also crosses the back of the knee but this calf muscle also works on the back of the ankle.)
The biceps-femoris-long-head originates at the sitting bone (ischial tuberosity), and then runs down the back of the thigh, connecting to the short head muscle and from there attaching with it to the fibula.
Both the semimembranosus and the semitendinosus also attach to the sitting bone and run down the back of the thigh but instead of attaching to the fibular they attach to the inside of the tibia just below the knee.
The semimembranosus is the "deeper" of the two muscles, meaning it is closer to the bone. It is also thinner and "wider" like a band. It attaches to the top of the inner-rear quarter of the tibia, just below the knee, perhaps making up part of the knee capsule.
The semitendinosus is actually thin and tendon like for about half it's length (the lower half). It curves around the back of the knee and then attaches to the inside of the tibia along with the tendons of the gracilis and sartorius. Because of the way that the tendons of these muscles curve around the back of the knee, with the knee straight and these muscles active, they create a forward push on the knee where they cross behind it but create a backwards pulling force on the inner edge of the tibia. This could be one mechanism via which the knee joint is stabilized while straight.
Standing on one leg with the other knee bent so that the shin points backwards, tension in any of these muscles could be used to rotate the shin inwards with respect to the thigh. Meanwhile, the biceps femoris muscles could be used to rotate the shin outwards with respect to the thigh.
Going back to the hip thrust exercise, after thrusting the hips forwards with the glutes active, I often instruct my students to also activate the hamstrings. The hamstrings and glutes then work together to bend the hip backwards (extension).
One idea that is important with respect to yoga anatomy is optimal muscle length.
The idea is that there is a certain length at which muscle tissue can comfortably operate. It's like putting up an old fashioned tent. Putting the pegs in for the tent lines, if you put the pegs to close to the tent then you can't add sufficient tension to the tent lines. If you place them too far away then the tent lines won't reach the pegs. With muscle function there may be particular joint positions where it is optimal for one set of muscles to activate because they are at optimal length versus another set of muscles.
As an example of this, drawing the ribs up away from the pelvis adds tension to the abdominal muscles making it easy to activate the abs with minimal effort.
With the hamstrings, bending the knees tends to give them more slack while straightening the knees removes slack. With knees straight bending the hips forwards further reduces slackness while bending the hips backwards increases slackness.
In a pose like bridge pose where the intent can be to push the pelvis upwards, causing the hips to bend backwards, the hamstrings will have too much slack to be able to effectively activate. And so the job of extending the hips in this pose will fall on the glute maximus and the adductor-magnus-long-head.
Note that to lower the hips under control the gluteus maximus and adductor magnus will have to stay active until the moment the pelvis touches the floor. Then they can relax. In this instance these muscles are lengthening while activated. It can be helpful to then think of these muscles as having the function of resisting hip flexion.
Forward bends are usually considered hamstring stretches. Carrying on with the idea of resisting hip flexion, the hamstrings and glutes can be useful as you bend forwards because they help to control the descent of the body.
In these instances these muscles work against the weight of the body. While bending forwards these muscles lengthen while contracted until the point that your hands touch the floor or your ribcage comes to rest against one or both thighs.
Moving in the opposite direction these muscles shorten while active to tilt the pelvis rearwards relative to the thighs.
For the gluteus maximus, optimal operating length is challenging to find in a pose like reverse plank (Purvottanasana), which is perhaps why so many yoga students find this pose less than fun.
I used to hate reverse plank too until I started activating my hamstrings prior to lifting my hips.
So that you can use the glute maximus in this pose you may find it helpful to tilt the pelvis forwards so that the back of the pelvis moves away from the backs of the knees. You may find that you can then use your glutes in this pose to help lift your pelvis.
Other poses where you may find it helpful to consciously activate the glutes and hamstrings are in poses like pigeon pose and high lunge. In both of these poses the intent can be to stretch the front of the hip of the rearwards reaching leg.
To that end, with the knee straight, focus on lifting the rear thigh in such a way that the glute and hamstrings of that leg activate. You may find that this action helps to push your pelvis downwards.
Having trouble activating your leg hamstrings and glutes? The Basic Muscle Control Quads, Hip Flexors, Hamstrings and Glutes video uses simple exercises to make it easy for you to learn how.
The book (and optional video) Muscle Control for Better Flexibility teaches you how to activate your hamstrings and glutes in the context of a yoga routine. It's great if you want to work out and improve muscle control at the same time.