I write code in both JAVA and C#. Even though I am currently doing allot of my programming in C#, I like both languages and switch between them to make sure I am equally satisfied with both languages, and don't gain a "weak" end from working on one programming language more than the other.
But really, this is my opinion:
If you only use windows, and have no plans on using anything else, you should start learning C# HEAVILY, as it is most likely a very vital language in the future for windows users.
If you plan on using other operating systems, then you should stick with JAVA. JAVA's functions aren't as powerful as C# is, since C# is directly and only for windows platforms.
Overall, I would suggest C# to a beginner. It's both easier to learn, and the functionality and msdn document library is amazingly helpful.