Is it justified to get with my friend's ex?

I've liked Ryan on and off ever since we were 13, although I have never told him how I felt until recently. When we were 16, his girlfriend at the time, Emily, cheated on him and left him for my then-boyfriend's best friend, Max. Then, because our boyfriends were friends, Emily and I started spending quite a lot of time together too and for about 6 months we were pretty close. Whenever she had an argument with Max, she would confide in me that she really missed Ryan and regretted what had happened between them. Happy in my own relationship and thinking that nothing would ever happen between Ryan and myself, I would genuinely encourage her to try and make things work again with him etc. 4 years ago, I broke up with my boyfriend and began hanging out with different friends, including Ryan. We have since become good friends and, during this time, I haven't really had any contact with Emily, other than the odd comment on Facebook etc. Over the past few months, I've spent a lot more time with Ryan and started to really like him again and we have kissed a couple of times. Although right now we are just friends and may never actually be anything more, the main thing that is holding me back from taking things further is the thought of Emily's reaction. She and I barely speak and she treated him horribly during their own relationship, but I know that she did really like him and would be hurt if we were together because of our past friendship, which makes me feel guilty. Am I in the wrong or would a relationship in this situation be justified?

Put the internet to work for you. via Personal Recipe 2629979