FFXIV 3.4 content: Tanks are the support role of the party in PvP
Just jump in. There has never been a better time to get involved in PvP than right now. 3.4 content has killed Ranked solo-queue Feast, for the moment, so the only thing queuing is 8v8 casual. No one takes that seriously, so you can try out whatever job you want and not feel pressured by details. The dueling deck has been pretty active, so you can test out your skills in a 1v1 setting. If you can find a vet PvP'er, they can tech you what's up. Just keep in mind that the dueling deck isn't lvl sync'd and not all the kinks have been worked out, so actual PvP instances will have different dmg scaling and a slightly different feel to it.
As for starting out, here's a beginner's checklist to get you started:
Pick a role, then a Job
First, try to decide what role you want to fill in PvP. Tanks are the support role of the party in PvP. Dps are the killers, and Healers keep everyone alive.
Pretty much every job is viable, but they all play very differently. Within the support role, for example, Plds excel at crowd control and defensive buffs; whereas, Wars excel mostly at offensive buffs and pressure plays. Likewise, each specific Dps sports its own ideal burst rotation (typically some variation of an opener that blows all of it's offence in one shot), though some Dps (such as Blm's) also have party utilities such as sleep. The more you learn about what your chosen job can do and how it's useful to the party, the bigger an asset you'll be in an actual match. Don't be afraid to experiment, either. Nothing is permanent.
Further, don't fret too much if you get overwhelmed. Unlike PvE, PvP requires you to be adaptable and think on the fly. The pace of the combat is MUCH faster, so it takes a lot of experience to get comfortable with even one job. As a result, even seasoned PvP'ers aren't perfect and often make mistakes, especially if they're trying something new. Be patient with yourself and keep a tough skin. You have a lot to learn, and experience will be your best teacher.
Learn your PvP tabs
Once you've come to a decision on which job you'd like to take into PvP, it then boils down to prep work. Your first order of business is to check out what your job can do. Go into the character tab and select PvP, then check out your PvP actions. Read them, and get to know them as well as their future upgrades. As you are totally new, you won't be able to upgrade any of them just yet, but once you rank up you'll get AP points and be able to get some upgrades. There's no specific order to how you do your upgrades, either, so you can plan ahead for things you want right off the bat.
Set up a Hotbar
Next you'll want to set up a PvP hotbar. Go into your actions, select PvP, and start customizing your own loud out. How you want to do that is up to you. I play on console, so I have an entire hotbar dedicated just to PvP actions which I've trained my fingers to flip to whenever I need one. That said, I also have a horizontal Hotbar kept up on my screen (which is non-functional on console) for my most valuable PvP/PvE actions. The only purpose it serves is to give me a way to keep an eye on my CD's, so I always know what's up and what's down without actually having to swap my active hotbar. Everyone has their own way of doing things, though. You'll have to experiment and get a feel for what works best for you. As long as you have access to those abilities, then you're good to go. Even if they're still at level 1, they're still useful, so don't forget them.
Practice. Practice. Practice.
How you practice will vary depending on your role, but everyone starts off with the basics. If you're a dps, look up your burst rotation (some of them can be found here in the forums if you search around a bit), then hit up a practice dummy till you get comfy with it. Or, conversely, jump right into a match. Not many people (hopefully no one) are going to complain about a fresh face in a casual 8v8 match, so there's no harm in trying. The end results also give you an accurate way of judging just how much dps you did. The more the better. Keep in mind, however, that the numbers you'll see in a 8v8 match do not reflect what is considered "good" in a real, ranked, 4v4 match. There's just too many players to be comparable.
Dueling is another way to get some 1v1 practice, especially if you happen to find a PvP vet on your job who can show you the ropes. For healers, as well, it's a good way to practice healing through a proper PvP burst. Find someone to kick your ass up and down the arena and try to stay alive. Once you've gotten decent at it, go into a match and see if you can keep SEVEN other people alive as well as yourself.
As for Tanks, Duels are good, but they're no where near the same as what's expected of you in an arena. Wars probably have the biggest similarity between dueling and their actual matches, so if you chose that job and want to practice your double Fell Cleave rotation then it's a good place to start. Plds, on the other hand, will have fun in duels but it won't teach you anything about your party utility. The only way to learn that (for both jobs) is to get into actual matches.
Again, though, I can't stress enough how different casual matches are to ranked matches. If you eventually decide to take yourself into a ranked match, expect a very different pace and style. 8v8 can also be a bit of a pain to learn in, because there's just so much more going on (kinda like the difference between a 4man dungeon and a 8 or 24 man). Just try to keep your cool and train your eyes to take it all in.
If you decide you like PvP and want to make an honest go at improving and sticking around, then you'll probably find yourself looking for ways to improve and asking a lot of questions. If there's something you don't know how to do, can't figure out, or just want advice on, don't be shy about asking questions here in the forums. There's a lot of little things that happen in the background of PvP (such as ability restrictions, face target functions, hotbar gimmicks, Crowd Control resistances, duration, and reset timers, etc) that most casual players won't know unless they read up on the patch notes or come here to ask questions. There's also a lot of little tips and tricks we can teach you as well.
After that you just gotta decide if it's for you or not. I'm not going to lie to you, PvP is naturally a competitive environment, so the crowd can be rough at times. This is especially so in ranked matches, where your shortcomings can cost someone their standing. There's also a lot of longstanding problems that SE has failed to address. That said, it can be incredibly fun and addictive if you give it an honest shot and keep a tough skin. Try to have some fun with it and see if it suits you.