They would be serving a different purpose. At least in my car, the wiring originally went through a very long route from battery/harness into fusebox area then out to console switch then back towards front of car before then heading rearward to the window motor. Closing the switch (up/down) allows the electrical signal to go that whole route and power the motor, but because of the long run the actual voltage delivered at the motor is less than what is coming out of the battery (your 12-13V). There's a long thread on this where someone (perhaps Scott) calculated the actual voltage drop and it's significant, which means less power to power an old motor with some resistance from grease, etc. Updating the motors and cleaning the track helps, but the relay allows a run directly from battery straight back to the motor - the shortest possible route. The switch in the console wires into the relay and essentially controls the gate within the relay either allowing power through or not. That wire run between switch and relay (both there and back) is what is electrically cut out of the original distance that had to be traversed between power to the motor so much less voltage drop. When I first had to deal with this I found these somewhat confusing but after you sit with it for a bit and study the diagrams it makes sense and they are pretty easy to wire in, especially since there is room there in the window area to actually hold the relay next to the motor.
The other issue with running the power through the window switches is that those switches are pretty flimsy, and the motors draw a lot of current. The switch itself will age as the contacts burn and corrode, and that will add further resistance.
For those who may not be fully versed in electrical stuff, Ohm's law states that
V=IR
V is voltage, I is current and R is resistance. There are a number of ways to look at the window circuit, but the simple answer here is that the entire circuit from the battery, through the fuse box, possibly though circuit breakers, down about 10 feet of wire to and from the switch, the switch, and then the motor all comprise resistive elements. The motor resistance doesn't change (much) over time. Since V is fixed (V=12v) then IR must be fixed, and if R goes up I must go down...
Ultimately it is the current that does the work in a motor, and because power varies as the SQUARE of current, as current goes down, power goes down somewhat faster...
The entire circuit was not a great design to start with, but over time as the switches age, and the window tracks get gummed up, it gets worse..
Here is the window motor thread at the place where the two-wire relay diagram is discussed:
Window Relays