In this paper, we report a design of high sensitivity Photonic Crystal Fiber (PCF) sensor with high birefringence and low confinement losses for liquid analyte sensing applications. The proposed PCF structures are designed with supplementary elliptical air holes in the core region vertically-shaped V-PCF and horizontally-shaped H-PCF. The full vectorial Finite Element Method (FEM) simulations performed to examine the sensitivity, the confinement losses, the effective refractive index and the modal birefringence features of the proposed elliptical air hole PCF structures. We show that the proposed PCF structures exhibit high relative sensitivity, high birefringence and low confinement losses simultaneously for various analytes.