When you want to assess the impact on the environment of the building materials you're thinking of choosing, embodied energy is a key measure. Using materials with high embodied energy is like having a big carbon emission balance before you've even started building.
The higher the embodied energy, the more energy has been consumed in the manufacture of the product, and the more carbon emissions have occurred.
Embodied energy is the energy consumed by all the processes that are associated with the manufacture of the product. It's one of the things that can be measured by undertaking what's called a life cycle analysis.
A life cycle analysis tries to take a holistic look at the environmental impact of a material by classifying them in terms of:
CSIRO research indicates that on average, the materials used to build a home have already used about 10 times the annual operating energy of the home. As a result, no matter how energy efficient the home is top operate, you're already way behind the eight ball.
Lightweight construction has a generally much lower embodied energy than construction systems that use masonry materials like brick.
For example:
This video shows how this NSW-based builder went to a lot of effort to use materials with low embodied energy.
The graph below shows the embodied energy of a range of common construction systems.

You can read more about low embodied energy lightweight materials in this article: embodied energy