It's a tough predicament, that is for sure. Remember that quitting or giving up is just the easy way out. Having a good successful relationship is not easy, can be very stressful and takes a lot of work. But I am not saying you have to stay.
First and foremost, you shouldn't just end something with someone you love (or potentially love) just because you are attracted to another girl. It's no secret the wiener's mind controls the brain but that is one big reason why divorce rates are as high as they are today. Sex on the first night turns to if she isn't giving me what I want, I will find someone who will. A bad series of events starts off on the wrong foot already. Not saying all that happened to you, just playing a scenario. You shouldn't stay with your current girlfriend if you see no future. But if there is the potential for a great future, you can't give up on it, no matter what kind of fights and break-ups you have had in the past. Here is why.
Ultimately it comes down to two things (among many): selfishness and forgiveness. People mess up, do stupid things, say awful things, lie, cheat, steal, have awful thoughts, etc and it takes a big person to apologize. It takes an even bigger person to forgive, wholeheartedly. We typically do those dumb things because we are being selfish. We want something, we want it now and we will do everything to get it, no matter the cost or who gets in the way. It's all about me and my needs, now. It's all over. Financial decisions all the way down to who can control the remote. We all do it.
But really, when in a relationship when it is going absolutely awesome you will find that you will give yourself entirely to the other person. You will be ridiculously unselfish. All of your personal needs will come second to theirs. You will do anything and everything for the other person and if they screw up, no matter the capacity you may be hurt but you are very forgiving because you know it's the best thing to do. When relationships are going bad, its generally the opposite and that is when we most need to not be thinking selfishly about our own needs but what can I do, what can i give (literally and figuratively) to make this work. Start a series of unselfish moves and you will be surprised how quickly and relatively easily the tides turn.
This is then where we have to remember that looks likely will fade over time. Sex happens regularly after marriage and naturally decreases in frequency over time. We start off loving by attraction and have to learn to evolve to love by relation. That is not an easy transition for guys who are driven by their wieners.
You started dating this girl probably for a number of reasons. And those reasons brought you this far. You had a connection that you still have and if you break that off, you should feel bad. It's like losing a close friend. It hurts. You definitely shouldn't do it only because you have an attraction to another woman. There will always be those "hot" women that catch your eye. It's inescapable, especially in todays photoshop world, LOL. You can't allow yourself to keep looking outside or you will be missing something awesome in front of you. But if it's not garnering a future that you are willing to give your all for, then it's probably best to move on. However sometimes it takes losing something great to realize what you had... and who knows you may end up reconnecting.
I have been with my wife for 10 years now, only married 8 months...I know, I know. Been there, heard that...9 years to get married? blah blah blah. But we we've been best friends since day 1. There is no perfect formula for a successful relationship and only you can determine yours but being very forgiving and very unselfish can only help. It took me a while to realize how selfish I was being in even the most menial things. This is where I hope I don't lose you. I realized this after watching a web series called The Peasant Princess. It's a Christian video sermon series by Mark Driscoll that we started watching in my small group. The intro is sorta my little pony-ish and the sermon gets into some pretty detailed relationship/marriage topics but whether you are religious or not it provides some very valuable insight as to how manage your relationship (manage being an imperfect word for the context but it's all I can think of at the moment) and really challenges you to be better in your role in the relationship. He even talks about that "7 year itch" kinda thing where it starts off great, then gets into that 7th-ish year where it's not so great and you want to kill each other and how those low points can really strengthen your relationship when you really put in the effort to forgive and be unselfish and work it out. And how after that it gets waaaay better but you still have those rough times, you just know how to handle them better.
I know the above is a bit disjointed in its flow but its off the cuff. And it's stuff I learned that I thought would help. I highly suggest everyone check out that Mark Driscoll series, religious or not, it's really good. Hope this helps. Sounds like you need some time to yourself to sit down with your thoughts and hash out your priorities and what you want your future to entail. Good luck, bro.