Objective Statistical models for medical images have been developed to increase robustness in the segmentation process. In this project, a fully automatic approach to build a statistical shape-intensity model and combine this model with level set segmentation was designed, implemented and tested by applying the algorithm to clinical image data. Methods By using a hierarchical registration approach based on mutual information and demons registration, 3D statistical shape-intensity models were created by applying Principal Component Analysis. Using these models in combination with level set segmentation results in a fully automatic modeling and segmentation pipeline. Results Examples for shape-intensity models were synthesized and these models were used to automatically segment 3D MRI and CT images. Quantitative evaluation of the framework was performed by comparing automatic segmentation results to segmentation results of medical experts. Conclusion Evaluation tests in which this method was used for the automatic segmentation of femora and cardiac MRI endocardial surfaces are very promising. The implementation of an additional cost function term and the addition of information about the surroundings of an organ in the model are currently under development.