Field Programmable Gate Arrays (FPGAs) are a promising platform for accelerating image processing as well as machine learning applications due to their parallel architecture, reconfigurability and energy-efficiency. However, programming such platforms can be quite cumbersome and time consuming compared to CPUs or GPUs. This presentation shows methods and tools for reducing the programming effort for image processing applications on FPGA-based systems. Our design methodology is based on the Open-VX standard and includes an open-source High Level Synthesis (HLS) library for generating image processing and neural network accelerators called HiFlipVX. The importance of such an approach is shown with application examples from different research projects.