What is so interesting about starship sizes?
My answer is that they are an important part of Star Trek's consistency. Other figures such as warp speed or phaser power often lead to pointless discussions because they are quite abstract. In contrast, a physical dimension is something much more definite because it can be determined without any knowledge about fictitious subspace physics or the rapid nadion effect. We just need to remembers that starships are built to accommodate humans or humanoids to recognize that size does matter.
In spite of everything, starship sizes are often unknown or, even worse, in dispute. The reason is that they are no real-world objects designed for and built in a certain size. There is a model (hardware or CGI) of each ship. The size of the "real" ship, however, depends not only on what the model looks like but also on how it is presented on screen. Kitbashed designs and scaling up and down an existing design often obstruct the size estimation. Basically, there are three possible references to determine the size of a starship:
The decks of Federation starships are usually around 3.5m tall, confirmed by the interior sets whose basic dimensions are the same for all Federation ships and by MSDs and deck measurements of those ships whose size and deck count is known. The clear height of rooms and corridors on a ship is some 2.5m, bearing in mind that 24th century humans are evidently not taller than in the 21st century. Andrew Probert assumed 10 feet between decks, 2 feet in between and hence a ceiling height of 8 feet. Rick Sternbach designed the deck centerlines to be 3m to 4m apart, with 0.5m in between to accommodate EPS and consumables conduits. This gives us up to 3.5m clear space, or it would allow Jefferies tubes between the decks. So unless we know how tall the decks are intended to be, there is an uncertainty range of some 25%. Moreover, this method doesn't work for alien vessels that often don't have window rows to count, let alone cross-sections to shed light on their internal structure. Even if it were possible to count the decks, the deck height is not necessarily the same as on Federation ships.
Using deck heights as a scale
Starship encounters or space battles are commonly used references, provided that trustworthy figures for at least one of the participating ships are known. However, when analyzing the visual effects of the pre-CGI age, the limitations of motion control photography have to be taken into account. Occasionally two ships of different scales were filmed in one take (like the Excelsior and Enterprise-D), moreover naturally at a much too close distance. Shuttles sometimes can't be made as small as they should be because they would be barely recognizable. But even in cases where it would be technically possible to get the relative scaling right and with modern CGI, VFX people sometimes don't do their job carefully enough or even increase or decrease ships intentionally for the dramatic impact. So the comparison method may yield contradictory results. A considerable error has to be taken into account, and often the only reliable statement is something like "the unknown ship X is bigger than ship A", or "ship X is at most or at least y meters long".
Evaluating starship encounters
Size figures in official publications or given by individuals involved in the production process (Probert, Okuda, Sternbach, Drexler, Eaves, Stipes) are possible references likewise. However, they should be treated judiciously, for they don't necessarily reflect the "actual" sizes of the ships. For instance, if the designer says his ship is x meters long, but the VFX people evidently always assume y meters, the whole thing has to be reconsidered. Moreover, ship sizes may be based on a preliminary sketch rather than the final ship, or they use a faulty reference, which is the case in the Deep Space Nine Technical Manual where, among other ships, the well-known Excelsior is scaled up by 10%. The sizes mentioned in several official publications are evaluated on a separate page.
A Classification System
Concerning the reliability of size figures and of the designs themselves, I have found the following categories of ships (however, I don't refer to them elsewhere at EAS because they are too abstract):
Type A A ship which has been designed in a process close to real engineering and is well-documented, implying that the designer has adhered to a certain size and deck count and the people in charge of the VFX abide by this size. There is at least a sketch of the interior layout in the form of an MSD or a similar cross-section, and it has to match the outer appearance as indicated by window rows and lifeboats. If such a ship is present in a scene with contradictory size ratios, it has to be used as a reference, rather than being scaled up or down itself. Such a design is what it should be, a scale model, i.e. with a fixed scale and a definite size. Only the major ships featured in the shows, the Constitution refit, Galaxy, Intrepid and NX classes are of this type, but rather not the Defiant and definitely not the "Star Trek (2009)" Enterprise.
Type B A ship which has not been designed quite as carefully as type A, but whose size can be exactly determined due to very reliable references. For instance, the ship employs the same components as another ship, or there is a size chart which has been used for both the design and the VFX. If this applies, the figures are as reliable as those of type A. The Ambassador, Excelsior, Miranda, Constellation, Sydney, Nebula, Prometheus, D7 and K't'inga are members of type B.
Type C A ship whose size is not quite as evident, but can be determined with a small uncertainty range. This is the case if the size based on the deck count, the size comparison with other ships, a possible figure provided by the designer and references from literature are close together. The error should be less than 10%. This type includes the Akira, Oberth, D'deridex, Vor'cha, Galor and the Ferengi Marauder.
Type D A ship whose size stays about the same in VFX shots, but a reliable figure is not available. Such a ship usually doesn't have window rows to count, and it seems it has been intentionally designed without a explicit size in mind. There are many alien ship types in this category, for instance the Romulan scout, Jem'Hadar "bug" and battlecruiser, (Bajoran) Antares-class freighter and Bajoran raider, as well as the Norway, Steamrunner and Saber.
Type E A ship whose perceived size is subject to vary and is in dispute, irrespective of the figures given in publications, by the designer or by the VFX staff. This is the case if a design changes its size (Defiant, Deep Space 9, "Star Trek (2009)" Enterprise) or if the same model is used to represent ships of intentionally different sizes (Klingon Bird-of-Prey, Kazon raider/fighter, Jem'Hadar battleship), both of which is impossible. The same applies for kitbashed designs combining scaled components such as the Yeager class.
Type F A ship whose size is not that relevant. These are alien-of-the-week ships for which the same model (the triangular guest starship from TNG, for instance) is used again and again. Other ships whose size doesn't matter that much are those which can be described as "just huge" (Borg cube, Kazon mothership) or "just tiny" (ships which are described as shuttles or single-seated).
Summarizing, there are reliable figures for type A and B ships. The sizes of type C ships are not exactly known, but the evidence is sufficient for a good estimation. Ships of type D are satisfactory in that there are no blatant inconsistencies, but fans have derived different figures all of which could be true. Essentially the same applies to the type F ships.
This leaves the type E ships as the ones that cause the tough problems because of the contradictory evidence. There is no solution but to find a suited compromise, or to take the average of the shown VFX sizes, as Boris S. suggested. For further points of view on this topic have a look at the Starships & Other Technology forum of the Flare Sci-Fi Forums.
Starship Size Table - a list of starship sizes as given in several publications
Starship Scaling - about starships that look the same but are (supposed to be) of different sizes
Fleet Charts - comparison of all ships which allow a reasonable size estimation
Fleet Chart Annotations - references used for the ship lengths listed at EAS
The New Enterprise Design - my two cents on the redesign in "Star Trek (2009)" and its true size
Defiant Problems - Just how little is the tough little ship?
The Bird-of-Prey Size Paradox - one, two or even more sizes?
Size of the Dominion Battleship - Does the 5km version really exist?
Thanks to Greg Price for the deck height references.