The network control plane may be forced to lower the bit rate per channel and the order of modulation format. A cognitive algorithm based on Fuzzy C Mean（FCM） technology is proposed to make learning and decision making in the Software Defined Optical Network（SDON）. First

the FCM algorithm is added to the SDON control plane to achieve better network performance than the non-cognitive control plane. Then

according to the transmission quality of the optical channel

the modulation format of the high-speed flexible rate repeater is determined independently in real time by using FCM algorithm. Finally

the performance of the FCM algorithm is evaluated through the computation of the long distance transmission network

which is then compared with the common Case Based Reasoning（CBR） algorithm in the optical cognitive network. The results show that FCM algorithm performs better in speediness and error avoidance compared with CBR algorithm.