Also, if it is just the keyboard you added, there should now be a key just to the left of the space bar with a world icon. Tapping on that will cycle through your different keyboards. Holding it down will list all the keyboards you have added.
But, if you accidentally set the International Language settings to Serbian, you might have trouble navigating the settings now, since everything will be in Serbian. That will also add the keyboard, but you might find that a minor inconvenience. If that is what you did try the following.
Open the Settings app. Seven or eight icons down you'll see one that looks like some gears with a silver background. The same icon as the Settings app, actually. Tap it.
Scroll to the bottom of the page. The forth item from the bottom should be the International settings. It's two above the battery Percentage (ON/OFF) switch. Tap on it.
The top item is Language. Tap it. You'll see a list of languages, that are happily listed in their native alphabets. Select the language you want.
The second item is the Keyboards. You can remove unwanted ones as Tanakamonitor described.