I recently made the switch from using Xsplit to using Open Broadcaster Software, as the last versions have really been impressing me - fixing a lot of the issues I've had before and giving me much better performance in games while streaming.
This is meant to be a super-quick introduction to how OBS works. I won't go too much in-depth in what you can tweak to improve your stream, but rather just a quick overview of how to set up the stream and how the UI works - and then you can figure out stuff for yourself later.
After installing and starting OBS, go to Settings. The "General" tab is pretty forgettable, so let's skip that part and get to the interesting bits.
Quality: This one is a bit tricky to explain, but basically you should lower this value the more motion there is on your stream. Quality 5 might be best for a Quake Live stream while quality 10 is probably better for streaming poker or slow-paced strategy games.
Max bitrate: This along with the quality setting decides how good-looking your stream is. This can't be higher than your upload - if you have 2Mbit upload, you should stay at around 1500 max. If you have 1Mbit upload, you should stay at around 700 max, and so on. Generally, you shouldn't go above 2000 as some viewers may not have enough download to handle your stream - or the streaming server can't provide you with the bandwidth.
Buffer: This can generally be the same as max bitrate, but I've found that setting it to the double of max bitrate reduces stream choppiness.
Audio codec is pretty unimportant. Audio bitrate should be at least 96, and anything above 192 is kinda pointless IMO - but the higher it is, the better sound quality.
Mode: Live stream means you're streaming, save to file means you'll just record locally.
FMS URL: Select the server closest to you geographically.
Play path/stream key: Quote from the official getting started guide...
Your Twitch.tv Stream Key can be found by logging in and going to http://twitch.tv/broadcast, then clicking Show Key. For Justin.tv it's http://www.justin.tv/broadcast/adv_other
For Own3d, it's a little bit more complicated. From the home page, click your username to open the dropdown menu and select My Livestreams. Find the stream on this list you wish to stream to and click Edit. On this page, next to FMS Stream you can click Show to get your Stream Key.
By checking "Save to file" further down the panel, you can save to a file at the same time as you're streaming.
You can leave base resolution most of the time. Resolution downscale is the setting you want to change - if you downscale more, you'll generally get better performance. Streaming at a higher FPS also impacts the performance.
Make sure to check "Disable Aero at startup". It helps A LOT, and I'm not sure why it isn't checked by default.
Here you can pick your microphone input and set push-to-talk and other hotkeys if you want to. Mic/Aux Boost will let you boost the microphone if you need to. Experiment with this using local recordings.
The only part here I can really say anything about is "x264 CPU preset". Basically, here you can trade CPU power for bandwidth. If you have low upload but a really good CPU, you can use more CPU to get better quality without increasing your bitrate.
"veryfast" is the default, and uses very little CPU. "fast" uses a bit more but you may notice a performance drop. "medium" and anything below will almost definitely impact your game perfomance, and I only use it if I'm not actually playing.
When you've finished fooling around with settings, it's time to set up a scene. Right-click and add a scene, give it a name, and let's add some sources.
This is the menu for adding sources. "Software capture" ("screen region" in xsplit) is the most important one, as it'll let you broadcast your monitor and non-fullscreen games. Game Capture will let you capture fullscreen games, but may cause FPS drops.
You can also add images, slide shows, text and more via plugins. To add a webcam, you need to add a "video capture device". If there are some sources you use in more than one source, consider adding them to global sources (there's a button for this in the bottom right) so you can quickly add them.
When you've added all the sources you want, click "Preview Stream". From here, you can resize and move sources by clicking the "Edit Sources" button. When you're ready to stream, end the preview and press "Start Streaming". Have fun!