You can only work tiles that are within three tiles of your actual city tile - even though the cultural borders of your city can expand to be five tiles away - meaning that any improvements on tiles that are four or five tiles away are not worthwhile, unless they are resource tiles. You do get the resources (both luxury and strategic) from improved tiles that are four or five tiles away, you just can't get the yields.