It's so that you can work up towards becoming a full member which gives you the option to add an avatar and signature. Along with posting in sections that you are not able to while in the probation group. Other reasons to why avatars are not allowed is that it maybe abused (it rarely happens but that's another reason). For example, a user could register, upload an advertising avatar (advertising their site or something similar) and then spam the site (another reason why there's captcha too).
I hope that helps.
P.S. The probation group will be reviewed soon, so if you'd like to suggest anything (or anyone else who went through the probation group when they first registered) post below.
Thanks.