This is one of the most common complaints of pastor/s wives in regards to their husband. It is true that pastor’s often get caught up with what they consider urgent matters and fail to see the necessary issues at home that must be dealt with.
It is essential that you not let the fact that you feel neglected by your husband go on indefinitely. It is like a sore that festers; it will come to head some day and maybe not in the right way.
It is imperative that, sooner better than later, that you express your feelings to him, communicate your own needs and the needs of the family. Do not be angry, accusatory or defensive.  
If there is no change and the situation is critical you may need to go to a close friend or relative (someone he respects) and ask for help. This may seem like a last-ditch effort but it would be better to do this, and risk his anger, than to find your self in an untenable position.  
Also consider that your husband does have a very hectic schedule and many matters to attend to. It is his calling, and job, to serve the people he pastors. So do not make unreasonable demands, give room for last minutes emergencies, but do deal with the situation immediately.  
Pastor’s wives have a very challenging role to play in a church. They are not considered part of the church staff, but their ministry is as valuable to the church as their husbands!