Water management and control in irrigation canals are challenging due to the nonlinear behavior of open-channel flow, time delays, and variable water withdrawals. In recent years, reinforcement learning has emerged as a promising approach for controlling such systems; however, the influence of the underlying hydraulic model on the quality of the learning process has not yet been quantitatively investigated. This study compares the performance of a discrete reinforcement learning algorithm integrated with two simulation environments: a nonlinear hydraulic model based on the full Saint-Venant equations and a linearized hydraulic model of an irrigation canal. The learning agent was trained independently using each model, and the resulting control policies were evaluated based on water delivery errors as well as the efficiency and adequacy performance indicators. The results show that the nonlinear model achieved substantially smaller delivery errors than the linear model in the downstream reaches, with minimum efficiency and adequacy values of 0.962 and 0.972, respectively. In contrast, the minimum efficiency of the linear model decreased to 0.497 in the terminal reaches, while the maximum absolute delivery error reached 59%. Although the linear model exhibited faster convergence during training and provided acceptable performance within the linearization range (±20% discharge variation), it learned weaker control policies outside this range, resulting in degraded water distribution performance, particularly in the downstream reaches.