Comparison of texture compressions
Many different compressions can be used in GTA V for .dds textures, they can be selected when editing textures in OpenIV. The most common one is DXT, which is used by default and gives the best compromise
between size and quality. In some cases however, DXT loses color information, which is especially noticeable with grey colors. For example, here's a 16x16 pixel texture, first compressed with DXT, then saved with ARGB with 8 bit per channel (A8R8G8B8 in OpenIV).
Please note that the size of dds files in Windows cannot be compared to the size of a .ytd texture file directly.
As slightly visible in the images of the textures, the DXT .dds file is slightly pink, while the size is a lot lower.
Which compression should I use in-game?
In almost all cases, especially with larger textures or colorful textures, DXT is the better option. However, for specific parts, exact colors are necessary. In that case, the best option is to use a really small texture to keep the size of the file low. If A8R8G8B8 is used for too many or large textures, it may cause disappearing textures and crashes.
The different numbers behind DXT are settings for alpha maps (transparency). DXT 1 is used for texture without any alpha channel (completely opaque). DXT 3 has an alpha map, but it does not allow smooth transitions, it can be used for example for textures with a plain alpha map, where each pixel has the same transparency. DXT 5 supports an interpolated alpha and can be used for smooth transistions.
Avoiding discolouration with DXT compression
Often when converting to DDS, choosing any of the DXT formats will cause the discolouration of some colours due to the compression of the texture and leave you with an awful looking purple or green shade hanging over your texture. For the sake of greyscale colours, I've drawn up a colour table ranging from R0, G0, B0 to R255, G255, B255 for testing purposes that I used to see exactly what colours would be altered after export to DDS.
As an example, what I found was that colours such as R41, G41, B41 would result in being R41, G42, B41. The green being a mere one above red and blue results in this green shade. Other examples are the inverse of that, such as R130, B130, G130 which results in R131, B130, G131 after compression. Because the green is lower in this case, the colour takes on the purple shade.
Comparison of tables before and after DXT compression
I've marked the colours on the table with ticks that resulted in equal RGB colours in the DDS.
Ideally, this table should apply to most textures but I can't guarantee that it will always work. Should you choose to use this table as reference though, refer to the table for the blocks with ticks in them. The marked blocks are colours that were rounded off to equal RGB values upon compression without any green or purple dominance.
To give you an example of how to interpret the table, refer to column Y=40 & X=7. Originally it was RGB 47 and after compression remained so, meaning you can use R47, G47, and B47 in your texture before saving to DXT. On the other hand, sometimes it appears to round off a number, an example of this is Y=20 & X=9 which should be RGB 29 but is not the case after compression as it results in R30, G30, B30. Regardless of the rounding, its still equal though and can be used but the key is to use the value before compression which was originally R29, G29, B29.
I want to reiterate though that I can't guarantee this will always work, but its worth playing around with.