An unmanned aerial vehicle (UAV) is an aircraft capable of sustained flight without a human operator on board which can be controlled either autonomously or remotely (e.g., by a pilot on the ground). In recent years, the unique capabilities of UAVs have attracted a great deal of attention for both civil and military application. UAVs can be controlled remotely by a crew miles away or by a pilot in the vicinity. Vision-based control (also called visual serving) refers to the technique that uses visual sensory feedback information to control the motion of a device. Advancements in fast image acquisition/processing tools have made vision-based control a powerful UAV control technique for various applications. This thesis aims to develop a vision-based control technique for two sample experimental platforms, including: (1) a 2-DOF (degrees of freedom) model helicopter and (2) a 6-DOF quadrotor (i.e. AR.Drone), and to characterize and analyze response of the system to the developed algorithms. For the case of 2-DOF, the behavior of the model helicopter is characterized and the response of the system to the control algorithm and image processing parameters are investigated. In this section of experiments, the key parameters (e.g., error clamping gain and image acquisition rate) are recognized and their effect on the model helicopter behavior is described. A simulator is also designed and developed in order to simplify working with the model helicopter. This simulator enables us to conduct a broad variety of tests with no concerns about the hardware failure or experimental limitations. It also can be used as a training tool for those who are not familiar with the device and can makethem ready for real-world experiments. The accuracy of the designed simulator is verified by comparing the results of real tests and simulated ones. A quintic polynomial trajectory planning algorithm is also developed using the aforementioned simulator so that servoing and tracking the moving object can be achieved in an optimal time. This prediction, planning and execution algorithm provides us with a feasible trajectory by considering all of the restrictions of the device. The necessity of re-planning is also addressed and all of the involved factors affecting operation of the algorithm are discussed. The vision-based control structure developed for the 6-DOF quadcopter provides the capability for fully autonomous flights including takeoff, landing, hovering and maneuvering. The objective is to servo and track an object while all 6 degrees of freedom are controlled by the vision-based controller. After taking off, the quadcopter searches for the object and hovers in the desired pose (position and direction) relative to that. In the case that the object cannot be found, the quadcopter will land automatically. A motion tracking system consists of a set of infrared cameras (i.e. OptiTrack system) mounted in the experiment environment, which is used to provide the accurate pose information of the markers on the quadcopter. By comparing the 3D position and direction of the AR.Drone relative to the object obtained by the vision-based structure and the information provided by the OptiTrack, the results of the developed algorithm are evaluated. Results of developed algorithms in this section provide a flexible and robust vision-based fully-autonomous controlled aerial platform for hovering, maneuvering, serving and tracking in small-size lab environments.