"if you use a passband filter around a target frequency, you average out many wave cycles of that frequency and all the information stored there is lost. "

This is not true; otherwise, FM radios wouldn't exist.

A bandpass filter that just gets your carrier and modulation through is what you should be designing to.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

Shannon and others formulated that a long time ago. It is about band-width of your filter. Modulation causes sidebands. Simple in AM and somewhat more complex in FM.
There is a limit to signal contents and carrier frequency. Your filter, however steep, sets the limit for upper modulation frequency.
A practical application is in AM radios. Google superheterodyn, intermediate frequency and band-width.

Gunnar Englund
www.gke.org
--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.

Thank you, IRstuff and Skogsgurra!

Now, I've seen the math of a FIR filter and also made a spreadsheet out of it. Apparently, it works fine. I tested it by submitting a signal with 6 different sine frequencies and after it goes through the filter, only what's within the passband remains.

But the math of the TIR filter is exactly what I described: the filtered signal intensities are calculated as a weighted average of previous points of the unfiltered signal, therefore mixing up intensities in different times.



For instance, let's think of a very simple AM encoding: half a cycle in full amplitude means 1 and half a cycle in 0 amplitude means 0. If I modulate 10101010101010 etc., my signal will be the positive halves of a sine. If I put that through the filter, it will average full amplitude semicycles with zero amplitude semicyles. The result is that all zero semicycles will be gone in the filtered signal. Not only that, lots of negative values also show up (there are both positive and negative weights, represented as b_i in the formula above).

Check out the graph. In the example I used 44100Hz as a sampling rate, a signal of 4900Hz modulated like above (just the positive halves of a sine, the rest is 0). In the filtered signal, all areas of "silence" disappeared, as the maths should have you expect.




Am I doing anything wrong?

Dear Skogsgurra

As I understand, the superhet make sense if you have more than one frequency. But the problem I mentioned arises for any number of arrier frequencies, including just 1. Or am I wrong here? Anyway, I do not intend to use more than one carrier frequency, and he problem remains. Do you have any idea how to solve it?

Thanks!

Your chopped sine wave has a frequency content much different to what you probably imagine.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

Dear GregLocock

You are right. Maybe that was not the best example I could have come up with. But anyway, whatever are the changes I make to a pure sine wave in the modulation process, they are going to introduce alien frequencies. I tried alternating full waves and blanks as well. Look what happenned.



I tried alternating frequencies. No mater what the signal is, the FIR summation averages out different wave cycles, thus scrambling the data.

So, how is this problem solved in practice?

"calculated as a weighted average of previous points of the unfiltered signal, therefore mixing up intensities in different times"

No, most FIRs are digital models of analog filters. Analog filters have phase delays that are dependent on the frequency; that's what you are seeing. "calculated as a weighted average of previous points" is not equivalent to "average out many wave cycles of that frequency and all the information stored there is lost." If you do the math correctly on each individual frequency you will see that the signal is preserved, but delayed.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

Thanks, IRstuff!

Yes, my description "average out many wave cycles" is not very accurate. I just meant that the weighted average of several input points across many cycles will inevitably smooth things out and scramble different cycles into a pastiche, and that is exactly what I see in my results.

Now, the math is very, very simple but of course I still could have made a mistake. Yet the filter does remove frequencies outside the passband, I checked. And if the signal is a pure sine wave, unaltered, it really goes unchanged through the filter (apart from some expected change in amplitude). If there is some mistake in the spreadsheet, are there any other tests I could perform to check that hypothesis?

What you call "simple modulation" involves multiplying by square wave functions that have frequency content at least 7 times your base frequency, which puts the higher frequencies outside of your passband.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

"How can we separate different frequencies in a signal in such a way that the information modulated into a given frequency is preserved? I mean, if you use a passband filter around a target frequency, you average out many wave cycles of that frequency and all the information stored there is lost. "

I think the thinking there is on the wrong track. If that statement is fundamental to your problems with this then you need to start over.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

IRStuff, I know!

The point is, ONLY a pure sine in a frequency f has only content in that frequency. Whatever modulated data is inserted into that pure sine creates higher frequencies. There is no way around this. So, the problem remains... How are different frequencies in a signal sorted out from each other? How do cell phones and tvs do it? They have several frequencies relatively close to each other.

Everything that can be done has already been mentioned; you either filter or you heterodyne.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

"Whatever modulated data is inserted into that pure sine creates higher frequencies."

And lower too. Called sidebands. Lower and Upper Sidebands.

Amplitude Modulation is quite straightforward (regarding sidebands), but Frequency and Phase Modulation just isn't.

Also, look up 'aliasing' in terms of sampling.

Signal processing, both analog and digital, is extremely non-trivial. It's not easy to provide little answers.

Quote:

It's quite simple stuff.

Nope.

Mike Halloran
Pembroke Pines, FL, USA

If you want to separate out frequency content then you want to work in the frequency domain, not the time domain. Fourier analysis/transform is the tool for this. If you are only interested in a single frequency, there are crude tools to get this (i.e. multiply your signal by a sinewave at that frequency and then low pass filter (but unfortunately you probably won't match the frequency/phase exactly so you'll have beating that you still need to deal with like you showed above)).

More detail of what signal you are specifically trying to 'pull out of the noise' will be helpful here.

Z

Thanks for the tips, VE1BLL!

Yeah, there are the sidebands! But do you think they interfere with the problem I have in hand?
Yes, sidebands might distort my signals if my filter passband is too narrow. But tat doesn't change the fact that the signal is distorted as it goes through the filter, right? Because that is the problem I am facing, no matter what kind of modulation I use.

zappedagain, maybe I haven't been clear enough.

I don't need the frequency content. I just want to demodulate a signal that comes with noise. Now, that is trivial... when tere is no noise. But the only way I know to eliminate noise is to use a filter (a passband one in this case, since I have noise both above and below the carrier frequency). And unless the signal is a pure sine (it never is when there is modulated data into it), the filter distorts it (see the FIR filter equation and the graphs above) and thus the data is lost.

Once again, if you want to to recover a modulated signal, then you need a passband that is at least as large as the largest frequency in the modulation that you are unwilling to sacrifice. The fact that you insist on using rectangular waveforms for the modulation means that you need something like 10x the carrier frequency for a plausible bandwidth.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

If you're doing this on a computer, as a simulation, then you should see if you can include an FFT module (sometimes called a 'Tool Kit' in MS-Excel) to allow you to *see* (with your eyeballs) the same information in the frequency spectrum domain. If you don't do this, then you're working blind. If you can do it, then you'll find that everything will fall into place much quicker.

In the RF world, one of my quips is that, "If RF were purple, then it wouldn't be considered magic." In other words, once you can see it, it's easy.

Same sort of thing applies when trying to do signal processing. You *must* work in the frequency spectrum domain most of the time, only occasionally referring to the amplitude versus time domain as required (example, pop-noise filtering where they're more visible in the amplitude plot).

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python (free) Octave(free)), or in the simplest case, Mathcad or SmathStudio, which are free, have the shortest learning curve and are pretty.

Ideally you'd give us a sample of the data you have in numerical form and what you hope to extract from it.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

I couldn't figure how to quote and still I'm trying to pull some complicated stuff...

Quote (IRstuff)

Once again, if you want to to recover a modulated signal, then you need a passband that is at least as large as the largest frequency in the modulation that you are unwilling to sacrifice. The fact that you insist on using rectangular waveforms for the modulation means that you need something like 10x the carrier frequency for a plausible bandwidth.

Right again, IRstuff! So my modulation of choice was not fit to my filter (or vice-versa). That is a very useful piece of information, thanks!

Let me give you some more info. My sampling rate is 44100Hz and I chose somewhat arbitrarily the carrier at 4900Hz (exactly 1/9 of the sampling rate). I'm dealing with sound being broadcast across a few meters around people and typical day-to-day noises, so I'd like the frequency to be as high as possible to be removed from the most important interferences, but at the same time I'd like to be as far away as possible from the 22050Hz limit so as not so sacrifice resolution. For now, 4900Hz is the compromise number, which we are free to change.

Back to our problem. I chose the amplitude rectangular modulation because it is easy to implement. But if my passband covers the 10x span from 4500Hz to 45000Hz (ok, as high as 22050 max), then I'll get all the unwanted noise above the carrier frequency, and I can't afford that.

All things said and done, it seems I need a modulation with the narrowest possible range of content in the frequency domain, right? Now, what would that modulation be?

Quote (VE1BLL)

If you're doing this on a computer, as a simulation, then you should see if you can include an FFT module (sometimes called a 'Tool Kit' in MS-Excel) to allow you to *see* (with your eyeballs) the same information in the frequency spectrum domain. If you don't do this, then you're working blind. If you can do it, then you'll find that everything will fall into place much quicker.

In the RF world, one of my quips is that, "If RF were purple, then it wouldn't be considered magic." In other words, once you can see it, it's easy.

Same sort of thing applies when trying to do signal processing. You *must* work in the frequency spectrum domain most of the time, only occasionally referring to the amplitude versus time domain as required (example, pop-noise filtering where they're more visible in the amplitude plot).

Yep, that's great advice, thanks! I just started some tests on Excel, which is my weapon of choice.
If I had done that from the start, I wouldn't have needed the insights IRstuff just gave me.

Quote (GregLocock)

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python (free) Octave(free)), or in the simplest case, Mathcad or SmathStudio, which are free, have the shortest learning curve and are pretty.

Yeah, maybe I should change. But I've been an Excel guy for such a long time...

Quote (GregLocock)

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python Ideally you'd give us a sample of the data you have in numerical form and what you hope to extract from it.

The data is going to be just a few bytes, a positive integer up to, dunno, 10^9.

Rather than try to bash some arbitrary solution into fitting your problem, perhaps you should start by describing what you functionally want to do, i.e., send how much data, how fast, how often, and how far. Then, and only then, perhaps you could consider looking at modem technology and how people were able to transmit up to 33 kbps over phone lines with a worst-case bandwidth of 3 kHz.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

Quote (IRstuff)

Rather than try to bash some arbitrary solution into fitting your problem, perhaps you should start by describing what you functionally want to do, i.e., send how much data, how fast, how often, and how far. Then, and only then, perhaps you could consider looking at modem technology and how people were able to transmit up to 33 kbps over phone lines with a worst-case bandwidth of 3 kHz.

Yeah, I used the 33kbps phone lines. But remember we had very little noise there! The need for filters is what makes this case more difficult and much closer to radio: an environment with a lot of noise as loud as your signal and in VERY close frequencies.

So, fair enough: I would like to send some 30 bits in less than 0.02s (or so) every couple of seconds across some 10 or 20m using a sampling rate of 44100Hz. That makes 1.5kbps, which seems pretty reasonable. The times I set are first approximations deriving from the fact that there could be multiple senders (maybe 10 or 20) which are impossible to coordinate, so their messages are spaced out across time (probably at random intervals up to a couple of seconds) in a way they don't superpose (too often). It's the only time-share solution I can envision in a situation you can't sync the senders.

So, I didn't do the math but I figured that if you send 30 different signals that are 0.02s each at 2 or 3s varying intervals, there is little superposition and in 5s you probably had at least one instance of most signals not superposed to another. The limiting factor here is obviously how long I am willing to wait to get most signals right, which I set in 4 or 5s. Of course, we could use 0.03s or even more if we have to. But I am hoping we don't.

"... I would like to send some 30 bits in less than 0.02s (or so) every couple of seconds across some 10 or 20m..."

There are plenty of off-the-shelf solutions for wee little radio links to perform such functions. The applicable modules are typically $1 or $2 from sellers on eBay. There's a whole ecosystem of this within the embedded "Arduino" community.

Edited to add: Andreas Spiess "The guy with the Swiss accent" on YouTube seems to specialize in this topic.

https://computers.tutsplus.com/tutorials/building-... etc

You could assign a different channel to each sensor, or you could increase the complexity and actually ping each sensor in turn and get it to answer back.

I have got half a dozen Arduino wifi boards, but the project I bought them for is fairly low priority, so apart from a quick look to see what they were like (they need aerisls for my application) I can't tell you much about the practical side.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

Quote (VE1BLL)

There are plenty of off-the-shelf solutions for wee little radio links to perform such functions.

But I'm doing it over sound. How could I apply that?

Quote (GregLocock )

You could assign a different channel to each sensor, or you could increase the complexity and actually ping each sensor in turn and get it to answer back.

But it is supposed to be a two way communication. All parties should be able to communicate with all others.

Over radio links, you could use TCP/IP, DNS, and DHCP.
... i.e., much of the hard networking stuff is already done.


Over sound links, maybe you could use time division multiplexing.
Divide each second into, say 32 periods.
Within all but one period, only one of up to ~30 transmitters can make noise; all listen.
Each sent message/squeak includes a few bits to designate the intended receiver,
a few bits to encode a message,
and a few bits to encode a checksum.
One address is for broadcast, message to everyone.
One timeslot is silent to synchronize everyone.
Everyone must send _some_ message within their designated time slot,
to show that that time slot is occupied and active,
so there must be a 'null message' for that purpose.
Identities 0..30 are preassigned to each unit.

You may need to add some fudge code to get all the transceivers into sync at startup,
like changing time delays from an arbitrary starting point until everyone receives
a complete cycle of messages with no checksum errors, i.e., no one is transmitting
when he shouldn't be.

Or equip everyone with radio receivers and use WWV for sync.

The above outline represents the core of something that might work,
BUT, if you are doing this among people,
expect complaints of annoyance about the buzzing if they can hear it,
and complaints of general malaise if they can't.

It's also going to be more expensive and more difficult than using stuff you can buy.

Mike Halloran
Pembroke Pines, FL, USA

Well the latest revelation rather knocks the entire project intio a different ballpark. If you're you have 2 way coms then collisions aren't an issue. I'm out of here until i see some sample data and a decent spec.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

Quote (MikeHalloran)

Over sound links, maybe you could use time division multiplexing.
Divide each second into, say 32 periods.

Interesting ideas, Mike. But I'd have to be able to sync everybody with a given clock first, which I'm not sure is possible. Anyways, time multiplexing is solvable. But I'll never get there if the signal doesn't make it through the filter, which is where I am right now.

Quote (GregLocock)

Well the latest revelation rather knocks the entire project intio a different ballpark.

Well, if y'all wanna help me along the whole project, that would be great! But I wouldnt ask that. All I'm asking is how to make a signal survive going through a filter.

Quote (GregLocock)

If you're you have 2 way coms then collisions aren't an issue.

I'm sorry, what are you talking about?

I didn't talk about the whole project before because most of its features are irrelevant and I thought those information would throw people away from the one thing I came here asking for help - and that's exactly what is happening now.

The fact that it is 2way com doesn't change the problem at hand: how to extract a signal in a noisy environment since filters tend to distort it. I'm guessing that TVs, cellphones and radios have filters with very little distortion because you can afford to use thousands of wavecycles to produce a single bit of information, so modulation functions can be very, very gradual, with derivatives very close to zero, thus being almost perfect sines at short intervals, and those kinds of signals have almost no sidebands and will go pretty much undisturbed through a filter.

In other words, I tried rectangular modulation functions with frequency once or two times the carrier frequency. That created huge sidebands, far outside my passband. On the other hand, TVs, cellphones and radios have gradual modulation functions with frequencies that are at least 2 or 3 orders of magnitude inferior to the carrier. Right now, I'm guessing that's where my problem lies. If that is true, the solution is to use more gradual modulation functions (and lose A LOT of bandwidth, of course).

I was hoping I could have 1 or two bits per wave. But apparently that is too much to ask.

"I'm guessing that TVs, cellphones and radios have filters with very little distortion because you can afford to use thousands of wavecycles to produce a single bit of information, so modulation functions can be very, very gradual, with derivatives very close to zero, thus being almost perfect sines at short intervals, and those kinds of signals have almost no sidebands and will go pretty much undisturbed through a filter."

No, that's not what they do. They do a systems engineering analysis, come up with rational and sound design requirements, and THEN, they design.

You keep harping about noise, but you have produced zero information about it. How can you expect to come up with a workable solution if you don't provide ALL the relevant design specs and environments?

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

Quote (Danielsp)

But I'd have to be able to sync everybody with a given clock first, ...

Read my msg again; I outlined two distinct ways to get the clocks in sync.

Mike Halloran
Pembroke Pines, FL, USA

IRstuff, I dont have any formal and/or fixed "specs"! That's just a personal project of mine. All I know about the noise is: it's environment sound. Could be virtually anything, but in all likelihood, it's just any public environment.
What relevant info is there about this that is needed to solve the problem? I really don't know!

The performance of such a system is dependent on the SIGNAL TO NOISE RATIO (SNR); if you don't know the noise, then you can't properly design the signal. You are basically wasting people's time with an ill-formed problem.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

You started with this: "I am a Civil Engineer who's just started in the field, so please bear with me here. It's quite simple stuff".

A few (quite a few, actually) tried to help you on the right track. But not easy with the very vague specifications that you presented. A moving target, as it turned out to be. My answer: "Shannon and others formulated that a long time ago. It is about band-width of your filter. Modulation causes sidebands. Simple in AM and somewhat more complex in FM". Was an attempt to make you understand the basic concepts. Did you google Shannon and his basic work?

If you think that "It's quite simple stuff" then you may have to rethink.

BTW, I think that I shall pick up bridge designing. There can't be much to it. Seems to be quite simple stuff. The Romans built the Pont du Gard and medieval masonrys built Cathedrals. So why shouldn't an electrical engineer like me start designing bridges? I wonder.

You may have noticed the little flag "Report" in the lower right corner. Well this thread is very close to being just that. And disappear.

Gunnar Englund
www.gke.org
--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.

Gunnar, you didn't do any recent work in Florida, did you?

Dan - Owner
http://www.Hi-TecDesigns.com

Not that I am aware about. S.C. is probably the closest I've left any marks. Not so sure what this is about. A political thing?

Gunnar Englund
www.gke.org
--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.

Dan was referring to the recent FIU pedestrian bridge collapse.
thread815-437029: Miami Pedestrian Bridge, Part V

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm

"...flag "Report" in the lower right corner. Well this thread is very close to being just that. And disappear."

Sigh...

Time and effort > /dev/null 2>&1

Perhaps we shouldn't bother at all.

No, Dan and IR. I haven't started my new career yet. But my teacher/mentor mentioned that he had been in Florida

VE1BLL, I am very liberal when it comes to impossible threads. But this one has not brought anything but confusion and frustration. We all learn, but the OP's conviction that "It's simple stuff' makes every effort go to the bit bucket.

Gunnar Englund
www.gke.org
--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.

If anyone finds that a given thread causes them frustration, then they're free to stop opening it. I take this approach all the time; it has much to recommend it.

The OP cannot fail to be learning from this thread. As well, some of the participants might have picked up a tidbit or two. And perhaps lurkers too.

It is not at all as simple as that, VE1BLL. And I think that you understand it, too.

Gunnar Englund
www.gke.org
--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.