inference - redwood.berkeley.edu

Inference - Wiener filter - coring/shrinkage

You observe y,what is x?

Simple example

y = x+ n

P (x|y) � P (y|x)P (x)

0 4 8 12 16 20 24

P

x, yobserved value

y = 14estimated value

x̂

prior

P (x) =1

Zxe� (x�µx)2

2�2x

<latexit sha1_base64="bOAisQ47RcwmY5A7sKVKgVGMJYM=">AAACIHicbZBNS8MwGMfT+Tbn29Sjl+IQtsNGO4R5EYZePE5wL7h2Jc3SLSxpS5JKR+lH8eJX8eJBEb3ppzHbetDNPwR++T/PQ/L83ZASIQ3jS8utrW9sbuW3Czu7e/sHxcOjjggijnAbBTTgPRcKTImP25JIinshx5C5FHfdyfWs3n3AXJDAv5PTENsMjnziEQSlspxio1WOK5eWxyFKzDS5d+IUD5LqwijHVYtFTlwZ1NOkbgkyYtCJ1SUtOMWSUTPm0lfBzKAEMrWc4qc1DFDEsC8RhUL0TSOUdgK5JIjitGBFAocQTeAI9xX6kGFhJ/MFU/1MOUPdC7g6vtTn7u+JBDIhpsxVnQzKsViuzcz/av1Iehd2QvwwkthHi4e8iOoy0Gdp6UPCMZJ0qgAiTtRfdTSGKhupMp2FYC6vvAqdes1UfHteal5lceTBCTgFZWCCBmiCG9ACbYDAI3gGr+BNe9JetHftY9Ga07KZY/BH2vcPGFKi7A==</latexit><latexit sha1_base64="bOAisQ47RcwmY5A7sKVKgVGMJYM=">AAACIHicbZBNS8MwGMfT+Tbn29Sjl+IQtsNGO4R5EYZePE5wL7h2Jc3SLSxpS5JKR+lH8eJX8eJBEb3ppzHbetDNPwR++T/PQ/L83ZASIQ3jS8utrW9sbuW3Czu7e/sHxcOjjggijnAbBTTgPRcKTImP25JIinshx5C5FHfdyfWs3n3AXJDAv5PTENsMjnziEQSlspxio1WOK5eWxyFKzDS5d+IUD5LqwijHVYtFTlwZ1NOkbgkyYtCJ1SUtOMWSUTPm0lfBzKAEMrWc4qc1DFDEsC8RhUL0TSOUdgK5JIjitGBFAocQTeAI9xX6kGFhJ/MFU/1MOUPdC7g6vtTn7u+JBDIhpsxVnQzKsViuzcz/av1Iehd2QvwwkthHi4e8iOoy0Gdp6UPCMZJ0qgAiTtRfdTSGKhupMp2FYC6vvAqdes1UfHteal5lceTBCTgFZWCCBmiCG9ACbYDAI3gGr+BNe9JetHftY9Ga07KZY/BH2vcPGFKi7A==</latexit><latexit sha1_base64="bOAisQ47RcwmY5A7sKVKgVGMJYM=">AAACIHicbZBNS8MwGMfT+Tbn29Sjl+IQtsNGO4R5EYZePE5wL7h2Jc3SLSxpS5JKR+lH8eJX8eJBEb3ppzHbetDNPwR++T/PQ/L83ZASIQ3jS8utrW9sbuW3Czu7e/sHxcOjjggijnAbBTTgPRcKTImP25JIinshx5C5FHfdyfWs3n3AXJDAv5PTENsMjnziEQSlspxio1WOK5eWxyFKzDS5d+IUD5LqwijHVYtFTlwZ1NOkbgkyYtCJ1SUtOMWSUTPm0lfBzKAEMrWc4qc1DFDEsC8RhUL0TSOUdgK5JIjitGBFAocQTeAI9xX6kGFhJ/MFU/1MOUPdC7g6vtTn7u+JBDIhpsxVnQzKsViuzcz/av1Iehd2QvwwkthHi4e8iOoy0Gdp6UPCMZJ0qgAiTtRfdTSGKhupMp2FYC6vvAqdes1UfHteal5lceTBCTgFZWCCBmiCG9ACbYDAI3gGr+BNe9JetHftY9Ga07KZY/BH2vcPGFKi7A==</latexit><latexit sha1_base64="bOAisQ47RcwmY5A7sKVKgVGMJYM=">AAACIHicbZBNS8MwGMfT+Tbn29Sjl+IQtsNGO4R5EYZePE5wL7h2Jc3SLSxpS5JKR+lH8eJX8eJBEb3ppzHbetDNPwR++T/PQ/L83ZASIQ3jS8utrW9sbuW3Czu7e/sHxcOjjggijnAbBTTgPRcKTImP25JIinshx5C5FHfdyfWs3n3AXJDAv5PTENsMjnziEQSlspxio1WOK5eWxyFKzDS5d+IUD5LqwijHVYtFTlwZ1NOkbgkyYtCJ1SUtOMWSUTPm0lfBzKAEMrWc4qc1DFDEsC8RhUL0TSOUdgK5JIjitGBFAocQTeAI9xX6kGFhJ/MFU/1MOUPdC7g6vtTn7u+JBDIhpsxVnQzKsViuzcz/av1Iehd2QvwwkthHi4e8iOoy0Gdp6UPCMZJ0qgAiTtRfdTSGKhupMp2FYC6vvAqdes1UfHteal5lceTBCTgFZWCCBmiCG9ACbYDAI3gGr+BNe9JetHftY9Ga07KZY/BH2vcPGFKi7A==</latexit>

likelihood

P (y|x) = 1

Zne� (y�x)2

2�2n

<latexit sha1_base64="pVvY6Or+J6MJ9Bli1v3e/p9grwE=">AAACHnicbZDLSsNAFIYnXmu9RV26CRahXbQkRdGNUHTjsoK9YJOGyXTSDp1MwsxEGmKexI2v4saFIoIrfRunl4W2/jDwzX/OYeb8XkSJkKb5rS0tr6yurec28ptb2zu7+t5+U4QxR7iBQhrytgcFpoThhiSS4nbEMQw8ilve8Gpcb91jLkjIbmUSYSeAfUZ8gqBUlquf1ovJw6h0YfscotTK0juXZbiblqdGMSmPSt1qllZtQfoBdJm6ZHlXL5gVcyJjEawZFMBMdVf/tHshigPMJKJQiI5lRtJJIZcEUZzl7VjgCKIh7OOOQgYDLJx0sl5mHCunZ/ghV4dJY+L+nkhhIEQSeKozgHIg5mtj879aJ5b+uZMSFsUSMzR9yI+pIUNjnJXRIxwjSRMFEHGi/mqgAVS5SJXoOARrfuVFaFYrluKbk0LtchZHDhyCI1AEFjgDNXAN6qABEHgEz+AVvGlP2ov2rn1MW5e02cwB+CPt6weQ+qId</latexit><latexit sha1_base64="pVvY6Or+J6MJ9Bli1v3e/p9grwE=">AAACHnicbZDLSsNAFIYnXmu9RV26CRahXbQkRdGNUHTjsoK9YJOGyXTSDp1MwsxEGmKexI2v4saFIoIrfRunl4W2/jDwzX/OYeb8XkSJkKb5rS0tr6yurec28ptb2zu7+t5+U4QxR7iBQhrytgcFpoThhiSS4nbEMQw8ilve8Gpcb91jLkjIbmUSYSeAfUZ8gqBUlquf1ovJw6h0YfscotTK0juXZbiblqdGMSmPSt1qllZtQfoBdJm6ZHlXL5gVcyJjEawZFMBMdVf/tHshigPMJKJQiI5lRtJJIZcEUZzl7VjgCKIh7OOOQgYDLJx0sl5mHCunZ/ghV4dJY+L+nkhhIEQSeKozgHIg5mtj879aJ5b+uZMSFsUSMzR9yI+pIUNjnJXRIxwjSRMFEHGi/mqgAVS5SJXoOARrfuVFaFYrluKbk0LtchZHDhyCI1AEFjgDNXAN6qABEHgEz+AVvGlP2ov2rn1MW5e02cwB+CPt6weQ+qId</latexit><latexit sha1_base64="pVvY6Or+J6MJ9Bli1v3e/p9grwE=">AAACHnicbZDLSsNAFIYnXmu9RV26CRahXbQkRdGNUHTjsoK9YJOGyXTSDp1MwsxEGmKexI2v4saFIoIrfRunl4W2/jDwzX/OYeb8XkSJkKb5rS0tr6yurec28ptb2zu7+t5+U4QxR7iBQhrytgcFpoThhiSS4nbEMQw8ilve8Gpcb91jLkjIbmUSYSeAfUZ8gqBUlquf1ovJw6h0YfscotTK0juXZbiblqdGMSmPSt1qllZtQfoBdJm6ZHlXL5gVcyJjEawZFMBMdVf/tHshigPMJKJQiI5lRtJJIZcEUZzl7VjgCKIh7OOOQgYDLJx0sl5mHCunZ/ghV4dJY+L+nkhhIEQSeKozgHIg5mtj879aJ5b+uZMSFsUSMzR9yI+pIUNjnJXRIxwjSRMFEHGi/mqgAVS5SJXoOARrfuVFaFYrluKbk0LtchZHDhyCI1AEFjgDNXAN6qABEHgEz+AVvGlP2ov2rn1MW5e02cwB+CPt6weQ+qId</latexit><latexit sha1_base64="pVvY6Or+J6MJ9Bli1v3e/p9grwE=">AAACHnicbZDLSsNAFIYnXmu9RV26CRahXbQkRdGNUHTjsoK9YJOGyXTSDp1MwsxEGmKexI2v4saFIoIrfRunl4W2/jDwzX/OYeb8XkSJkKb5rS0tr6yurec28ptb2zu7+t5+U4QxR7iBQhrytgcFpoThhiSS4nbEMQw8ilve8Gpcb91jLkjIbmUSYSeAfUZ8gqBUlquf1ovJw6h0YfscotTK0juXZbiblqdGMSmPSt1qllZtQfoBdJm6ZHlXL5gVcyJjEawZFMBMdVf/tHshigPMJKJQiI5lRtJJIZcEUZzl7VjgCKIh7OOOQgYDLJx0sl5mHCunZ/ghV4dJY+L+nkhhIEQSeKozgHIg5mtj879aJ5b+uZMSFsUSMzR9yI+pIUNjnJXRIxwjSRMFEHGi/mqgAVS5SJXoOARrfuVFaFYrluKbk0LtchZHDhyCI1AEFjgDNXAN6qABEHgEz+AVvGlP2ov2rn1MW5e02cwB+CPt6weQ+qId</latexit>

P (x|y) � P (y|x)P (x)

How to compute ?

=1

Zne� (y�x)2

2�2n

1

Zxe� (x�µx)2

2�2x

<latexit sha1_base64="OIadET3ve5/rHOwzQHQfOj1Wlxk=">AAACTHicbVBLSwMxGMzWV62vqkcvi0WoYMtuEfQiFL14rGAf2G2XbJptQ5PskmSlZdkf6MWDN3+FFw+KCGbbHvpwIDCZb4YvGS+kRCrLejcya+sbm1vZ7dzO7t7+Qf7wqCGDSCBcRwENRMuDElPCcV0RRXErFBgyj+KmN7xL581nLCQJ+KMah7jDYJ8TnyCotOTm0Y3jC4hiO4mfXJ7gblyaCsVxaXTerSRxxZGkz6DL9SVxLnJz/tG8f1RyWOQuZkZpJufmC1bZmsBcJfaMFMAMNTf/5vQCFDHMFaJQyrZthaoTQ6EIojjJOZHEIURD2MdtTTlkWHbiSRmJeaaVnukHQh+uzIk6n4ghk3LMPO1kUA3k8iwV/5u1I+Vfd2LCw0hhjqaL/IiaKjDTZs0eERgpOtYEIkH0W000gLobpftPS7CXv7xKGpWyrfnDZaF6O6sjC07AKSgCG1yBKrgHNVAHCLyAD/AFvo1X49P4MX6n1owxyxyDBWQ2/wCIobQp</latexit><latexit sha1_base64="OIadET3ve5/rHOwzQHQfOj1Wlxk=">AAACTHicbVBLSwMxGMzWV62vqkcvi0WoYMtuEfQiFL14rGAf2G2XbJptQ5PskmSlZdkf6MWDN3+FFw+KCGbbHvpwIDCZb4YvGS+kRCrLejcya+sbm1vZ7dzO7t7+Qf7wqCGDSCBcRwENRMuDElPCcV0RRXErFBgyj+KmN7xL581nLCQJ+KMah7jDYJ8TnyCotOTm0Y3jC4hiO4mfXJ7gblyaCsVxaXTerSRxxZGkz6DL9SVxLnJz/tG8f1RyWOQuZkZpJufmC1bZmsBcJfaMFMAMNTf/5vQCFDHMFaJQyrZthaoTQ6EIojjJOZHEIURD2MdtTTlkWHbiSRmJeaaVnukHQh+uzIk6n4ghk3LMPO1kUA3k8iwV/5u1I+Vfd2LCw0hhjqaL/IiaKjDTZs0eERgpOtYEIkH0W000gLobpftPS7CXv7xKGpWyrfnDZaF6O6sjC07AKSgCG1yBKrgHNVAHCLyAD/AFvo1X49P4MX6n1owxyxyDBWQ2/wCIobQp</latexit><latexit sha1_base64="OIadET3ve5/rHOwzQHQfOj1Wlxk=">AAACTHicbVBLSwMxGMzWV62vqkcvi0WoYMtuEfQiFL14rGAf2G2XbJptQ5PskmSlZdkf6MWDN3+FFw+KCGbbHvpwIDCZb4YvGS+kRCrLejcya+sbm1vZ7dzO7t7+Qf7wqCGDSCBcRwENRMuDElPCcV0RRXErFBgyj+KmN7xL581nLCQJ+KMah7jDYJ8TnyCotOTm0Y3jC4hiO4mfXJ7gblyaCsVxaXTerSRxxZGkz6DL9SVxLnJz/tG8f1RyWOQuZkZpJufmC1bZmsBcJfaMFMAMNTf/5vQCFDHMFaJQyrZthaoTQ6EIojjJOZHEIURD2MdtTTlkWHbiSRmJeaaVnukHQh+uzIk6n4ghk3LMPO1kUA3k8iwV/5u1I+Vfd2LCw0hhjqaL/IiaKjDTZs0eERgpOtYEIkH0W000gLobpftPS7CXv7xKGpWyrfnDZaF6O6sjC07AKSgCG1yBKrgHNVAHCLyAD/AFvo1X49P4MX6n1owxyxyDBWQ2/wCIobQp</latexit><latexit sha1_base64="OIadET3ve5/rHOwzQHQfOj1Wlxk=">AAACTHicbVBLSwMxGMzWV62vqkcvi0WoYMtuEfQiFL14rGAf2G2XbJptQ5PskmSlZdkf6MWDN3+FFw+KCGbbHvpwIDCZb4YvGS+kRCrLejcya+sbm1vZ7dzO7t7+Qf7wqCGDSCBcRwENRMuDElPCcV0RRXErFBgyj+KmN7xL581nLCQJ+KMah7jDYJ8TnyCotOTm0Y3jC4hiO4mfXJ7gblyaCsVxaXTerSRxxZGkz6DL9SVxLnJz/tG8f1RyWOQuZkZpJufmC1bZmsBcJfaMFMAMNTf/5vQCFDHMFaJQyrZthaoTQ6EIojjJOZHEIURD2MdtTTlkWHbiSRmJeaaVnukHQh+uzIk6n4ghk3LMPO1kUA3k8iwV/5u1I+Vfd2LCw0hhjqaL/IiaKjDTZs0eERgpOtYEIkH0W000gLobpftPS7CXv7xKGpWyrfnDZaF6O6sjC07AKSgCG1yBKrgHNVAHCLyAD/AFvo1X49P4MX6n1owxyxyDBWQ2/wCIobQp</latexit>

� logP (x|y) = (y � x)2

2�2n

+(x� µx)2

2�2x

+ const.<latexit sha1_base64="Xiw52cekiagCiuvkb7M4kTQSOb0=">AAACSXicbVBLSwMxGMzW9/qqevQSLEJFWnaLoBdB9OKxglWhW9dsmq3BPJYkK7us/XtevHnzP3jxoIgn07oHXwOB+WbmI8lECaPaeN6TU5mYnJqemZ1z5xcWl5arK6tnWqYKkw6WTKqLCGnCqCAdQw0jF4kiiEeMnEc3RyP//JYoTaU4NXlCehwNBI0pRsZKYfWqETA5gO16dpdvwf0gVggX9byRbV22hkUr0HTAUSjssO2WZtYIeBr+DGR2gNsw4JHMikBxiKXQpjl03bBa85reGPAv8UtSAyXaYfUx6EucciIMZkjrru8lplcgZShmZOgGqSYJwjdoQLqWCsSJ7hXjJoZw0yp9GEtljzBwrH7fKBDXOueRTXJkrvVvbyT+53VTE+/1CiqS1BCBvy6KUwaNhKNaYZ8qgg3LLUFYUftWiK+R7cvY8kcl+L+//JectZq+5Sc7tYPDso5ZsA42QB34YBccgGPQBh2AwT14Bq/gzXlwXpx35+MrWnHKnTXwA5WJT/bMsJw=</latexit><latexit sha1_base64="Xiw52cekiagCiuvkb7M4kTQSOb0=">AAACSXicbVBLSwMxGMzW9/qqevQSLEJFWnaLoBdB9OKxglWhW9dsmq3BPJYkK7us/XtevHnzP3jxoIgn07oHXwOB+WbmI8lECaPaeN6TU5mYnJqemZ1z5xcWl5arK6tnWqYKkw6WTKqLCGnCqCAdQw0jF4kiiEeMnEc3RyP//JYoTaU4NXlCehwNBI0pRsZKYfWqETA5gO16dpdvwf0gVggX9byRbV22hkUr0HTAUSjssO2WZtYIeBr+DGR2gNsw4JHMikBxiKXQpjl03bBa85reGPAv8UtSAyXaYfUx6EucciIMZkjrru8lplcgZShmZOgGqSYJwjdoQLqWCsSJ7hXjJoZw0yp9GEtljzBwrH7fKBDXOueRTXJkrvVvbyT+53VTE+/1CiqS1BCBvy6KUwaNhKNaYZ8qgg3LLUFYUftWiK+R7cvY8kcl+L+//JectZq+5Sc7tYPDso5ZsA42QB34YBccgGPQBh2AwT14Bq/gzXlwXpx35+MrWnHKnTXwA5WJT/bMsJw=</latexit><latexit sha1_base64="Xiw52cekiagCiuvkb7M4kTQSOb0=">AAACSXicbVBLSwMxGMzW9/qqevQSLEJFWnaLoBdB9OKxglWhW9dsmq3BPJYkK7us/XtevHnzP3jxoIgn07oHXwOB+WbmI8lECaPaeN6TU5mYnJqemZ1z5xcWl5arK6tnWqYKkw6WTKqLCGnCqCAdQw0jF4kiiEeMnEc3RyP//JYoTaU4NXlCehwNBI0pRsZKYfWqETA5gO16dpdvwf0gVggX9byRbV22hkUr0HTAUSjssO2WZtYIeBr+DGR2gNsw4JHMikBxiKXQpjl03bBa85reGPAv8UtSAyXaYfUx6EucciIMZkjrru8lplcgZShmZOgGqSYJwjdoQLqWCsSJ7hXjJoZw0yp9GEtljzBwrH7fKBDXOueRTXJkrvVvbyT+53VTE+/1CiqS1BCBvy6KUwaNhKNaYZ8qgg3LLUFYUftWiK+R7cvY8kcl+L+//JectZq+5Sc7tYPDso5ZsA42QB34YBccgGPQBh2AwT14Bq/gzXlwXpx35+MrWnHKnTXwA5WJT/bMsJw=</latexit><latexit sha1_base64="Xiw52cekiagCiuvkb7M4kTQSOb0=">AAACSXicbVBLSwMxGMzW9/qqevQSLEJFWnaLoBdB9OKxglWhW9dsmq3BPJYkK7us/XtevHnzP3jxoIgn07oHXwOB+WbmI8lECaPaeN6TU5mYnJqemZ1z5xcWl5arK6tnWqYKkw6WTKqLCGnCqCAdQw0jF4kiiEeMnEc3RyP//JYoTaU4NXlCehwNBI0pRsZKYfWqETA5gO16dpdvwf0gVggX9byRbV22hkUr0HTAUSjssO2WZtYIeBr+DGR2gNsw4JHMikBxiKXQpjl03bBa85reGPAv8UtSAyXaYfUx6EucciIMZkjrru8lplcgZShmZOgGqSYJwjdoQLqWCsSJ7hXjJoZw0yp9GEtljzBwrH7fKBDXOueRTXJkrvVvbyT+53VTE+/1CiqS1BCBvy6KUwaNhKNaYZ8qgg3LLUFYUftWiK+R7cvY8kcl+L+//JectZq+5Sc7tYPDso5ZsA42QB34YBccgGPQBh2AwT14Bq/gzXlwXpx35+MrWnHKnTXwA5WJT/bMsJw=</latexit>

� @

@xlogP (x|y) = � (y � x)

�2n

+(x� µx)

�2x

= 0<latexit sha1_base64="bSDqQRu07buEYTpI6OjxVht7hB4=">AAACTnicbVFNS8MwGE7n15xfU49egkOYyKQdgl4E0YvHCU4H6yxplm7BJC1JKi21v9CLePNnePGgiGazyHS+EHh4PsibJ37EqNK2/WyVZmbn5hfKi5Wl5ZXVter6xpUKY4lJG4cslB0fKcKoIG1NNSOdSBLEfUau/duzkX59R6SiobjUaUR6HA0EDShG2lBelTTcQCKcuRGSmiKW/yCY5C4LB7BVT+7TXXgMC2c9bSS7xqbogCNP3DTzvUqhJA2Xx96EmhjVJG2vWrP37fHAaeAUoAaKaXnVJ7cf4pgToTFDSnUdO9K9bLQZZiSvuLEiEcK3aEC6BgrEiepl4zpyuGOYPgxCaY7QcMxOJjLElUq5b5wc6aH6q43I/7RurIOjXkZFFGsi8PdFQcygDuGoW9inkmDNUgMQltTsCvEQmWq0+YGKKcH5++RpcNXcdwy+OKidnBZ1lMEW2AZ14IBDcALOQQu0AQYP4AW8gXfr0Xq1PqzPb2vJKjKb4NeUyl/opbQd</latexit><latexit sha1_base64="bSDqQRu07buEYTpI6OjxVht7hB4=">AAACTnicbVFNS8MwGE7n15xfU49egkOYyKQdgl4E0YvHCU4H6yxplm7BJC1JKi21v9CLePNnePGgiGazyHS+EHh4PsibJ37EqNK2/WyVZmbn5hfKi5Wl5ZXVter6xpUKY4lJG4cslB0fKcKoIG1NNSOdSBLEfUau/duzkX59R6SiobjUaUR6HA0EDShG2lBelTTcQCKcuRGSmiKW/yCY5C4LB7BVT+7TXXgMC2c9bSS7xqbogCNP3DTzvUqhJA2Xx96EmhjVJG2vWrP37fHAaeAUoAaKaXnVJ7cf4pgToTFDSnUdO9K9bLQZZiSvuLEiEcK3aEC6BgrEiepl4zpyuGOYPgxCaY7QcMxOJjLElUq5b5wc6aH6q43I/7RurIOjXkZFFGsi8PdFQcygDuGoW9inkmDNUgMQltTsCvEQmWq0+YGKKcH5++RpcNXcdwy+OKidnBZ1lMEW2AZ14IBDcALOQQu0AQYP4AW8gXfr0Xq1PqzPb2vJKjKb4NeUyl/opbQd</latexit><latexit sha1_base64="bSDqQRu07buEYTpI6OjxVht7hB4=">AAACTnicbVFNS8MwGE7n15xfU49egkOYyKQdgl4E0YvHCU4H6yxplm7BJC1JKi21v9CLePNnePGgiGazyHS+EHh4PsibJ37EqNK2/WyVZmbn5hfKi5Wl5ZXVter6xpUKY4lJG4cslB0fKcKoIG1NNSOdSBLEfUau/duzkX59R6SiobjUaUR6HA0EDShG2lBelTTcQCKcuRGSmiKW/yCY5C4LB7BVT+7TXXgMC2c9bSS7xqbogCNP3DTzvUqhJA2Xx96EmhjVJG2vWrP37fHAaeAUoAaKaXnVJ7cf4pgToTFDSnUdO9K9bLQZZiSvuLEiEcK3aEC6BgrEiepl4zpyuGOYPgxCaY7QcMxOJjLElUq5b5wc6aH6q43I/7RurIOjXkZFFGsi8PdFQcygDuGoW9inkmDNUgMQltTsCvEQmWq0+YGKKcH5++RpcNXcdwy+OKidnBZ1lMEW2AZ14IBDcALOQQu0AQYP4AW8gXfr0Xq1PqzPb2vJKjKb4NeUyl/opbQd</latexit><latexit sha1_base64="bSDqQRu07buEYTpI6OjxVht7hB4=">AAACTnicbVFNS8MwGE7n15xfU49egkOYyKQdgl4E0YvHCU4H6yxplm7BJC1JKi21v9CLePNnePGgiGazyHS+EHh4PsibJ37EqNK2/WyVZmbn5hfKi5Wl5ZXVter6xpUKY4lJG4cslB0fKcKoIG1NNSOdSBLEfUau/duzkX59R6SiobjUaUR6HA0EDShG2lBelTTcQCKcuRGSmiKW/yCY5C4LB7BVT+7TXXgMC2c9bSS7xqbogCNP3DTzvUqhJA2Xx96EmhjVJG2vWrP37fHAaeAUoAaKaXnVJ7cf4pgToTFDSnUdO9K9bLQZZiSvuLEiEcK3aEC6BgrEiepl4zpyuGOYPgxCaY7QcMxOJjLElUq5b5wc6aH6q43I/7RurIOjXkZFFGsi8PdFQcygDuGoW9inkmDNUgMQltTsCvEQmWq0+YGKKcH5++RpcNXcdwy+OKidnBZ1lMEW2AZ14IBDcALOQQu0AQYP4AW8gXfr0Xq1PqzPb2vJKjKb4NeUyl/opbQd</latexit>

x̂<latexit sha1_base64="KKaXq1DRH3bprdfdkqAYBXzc5rw=">AAAB73icbZBNS8NAEIYnftb6VfXoZbEInkoigh6LXjxWsB/QhrLZbtqlm03cnYgl9E948aCIV/+ON/+NmzYHbX1h4eGdGXbmDRIpDLrut7Oyura+sVnaKm/v7O7tVw4OWyZONeNNFstYdwJquBSKN1Gg5J1EcxoFkreD8U1ebz9ybUSs7nGScD+iQyVCwShaq9MbUcyepuV+perW3JnIMngFVKFQo1/56g1ilkZcIZPUmK7nJuhnVKNgkk/LvdTwhLIxHfKuRUUjbvxstu+UnFpnQMJY26eQzNzfExmNjJlEge2MKI7MYi03/6t1Uwyv/EyoJEWu2PyjMJUEY5IfTwZCc4ZyYoEyLeyuhI2opgxtRHkI3uLJy9A6r3mW7y6q9esijhIcwwmcgQeXUIdbaEATGEh4hld4cx6cF+fd+Zi3rjjFzBH8kfP5A+dfj90=</latexit><latexit sha1_base64="KKaXq1DRH3bprdfdkqAYBXzc5rw=">AAAB73icbZBNS8NAEIYnftb6VfXoZbEInkoigh6LXjxWsB/QhrLZbtqlm03cnYgl9E948aCIV/+ON/+NmzYHbX1h4eGdGXbmDRIpDLrut7Oyura+sVnaKm/v7O7tVw4OWyZONeNNFstYdwJquBSKN1Gg5J1EcxoFkreD8U1ebz9ybUSs7nGScD+iQyVCwShaq9MbUcyepuV+perW3JnIMngFVKFQo1/56g1ilkZcIZPUmK7nJuhnVKNgkk/LvdTwhLIxHfKuRUUjbvxstu+UnFpnQMJY26eQzNzfExmNjJlEge2MKI7MYi03/6t1Uwyv/EyoJEWu2PyjMJUEY5IfTwZCc4ZyYoEyLeyuhI2opgxtRHkI3uLJy9A6r3mW7y6q9esijhIcwwmcgQeXUIdbaEATGEh4hld4cx6cF+fd+Zi3rjjFzBH8kfP5A+dfj90=</latexit><latexit sha1_base64="KKaXq1DRH3bprdfdkqAYBXzc5rw=">AAAB73icbZBNS8NAEIYnftb6VfXoZbEInkoigh6LXjxWsB/QhrLZbtqlm03cnYgl9E948aCIV/+ON/+NmzYHbX1h4eGdGXbmDRIpDLrut7Oyura+sVnaKm/v7O7tVw4OWyZONeNNFstYdwJquBSKN1Gg5J1EcxoFkreD8U1ebz9ybUSs7nGScD+iQyVCwShaq9MbUcyepuV+perW3JnIMngFVKFQo1/56g1ilkZcIZPUmK7nJuhnVKNgkk/LvdTwhLIxHfKuRUUjbvxstu+UnFpnQMJY26eQzNzfExmNjJlEge2MKI7MYi03/6t1Uwyv/EyoJEWu2PyjMJUEY5IfTwZCc4ZyYoEyLeyuhI2opgxtRHkI3uLJy9A6r3mW7y6q9esijhIcwwmcgQeXUIdbaEATGEh4hld4cx6cF+fd+Zi3rjjFzBH8kfP5A+dfj90=</latexit><latexit sha1_base64="KKaXq1DRH3bprdfdkqAYBXzc5rw=">AAAB73icbZBNS8NAEIYnftb6VfXoZbEInkoigh6LXjxWsB/QhrLZbtqlm03cnYgl9E948aCIV/+ON/+NmzYHbX1h4eGdGXbmDRIpDLrut7Oyura+sVnaKm/v7O7tVw4OWyZONeNNFstYdwJquBSKN1Gg5J1EcxoFkreD8U1ebz9ybUSs7nGScD+iQyVCwShaq9MbUcyepuV+perW3JnIMngFVKFQo1/56g1ilkZcIZPUmK7nJuhnVKNgkk/LvdTwhLIxHfKuRUUjbvxstu+UnFpnQMJY26eQzNzfExmNjJlEge2MKI7MYi03/6t1Uwyv/EyoJEWu2PyjMJUEY5IfTwZCc4ZyYoEyLeyuhI2opgxtRHkI3uLJy9A6r3mW7y6q9esijhIcwwmcgQeXUIdbaEATGEh4hld4cx6cF+fd+Zi3rjjFzBH8kfP5A+dfj90=</latexit>

) x̂ =�2x y + �2

n µx

�2x + �2

n<latexit sha1_base64="54T9utMlH0S3Bbo4vShYLLYh0qU=">AAACQnicbVDLShxBFK02D3XMY5Is3RQZBMEg3RJIQAKSbLJUcXwwNTa3a6pnCquqm6rbZoamvy0bv8BdPiAbF4q4dWH12IvxcaDg3HPOpapOkivpMAz/BXMvXr56Pb+w2Fp68/bd+/aHj/suKywXXZ6pzB4m4ISSRnRRohKHuRWgEyUOkpNftX9wKqyTmdnDSS76GoZGppIDeiluH7FdORwhWJv9oWyTbVI2AizHFf1BWWqBl8zJoYZ4fLzBvkzWmsnUE9NFPK5mAjNu1YrbnXA9nII+JVFDOqTBdtw+Z4OMF1oY5Aqc60Vhjv0SLEquRNVihRM58BMYip6nBrRw/XJaQUVXvDKgaWb9MUin6uxGCdq5iU58UgOO3GOvFp/zegWm3/ulNHmBwvD7i9JCUcxo3ScdSCs4qoknwK30b6V8BL449K3XJUSPv/yU7G+sR57vfO1s/WzqWCDL5DNZJRH5RrbIb7JNuoSTv+Q/uSRXwVlwEVwHN/fRuaDZ+UQeILi9A4hdsMA=</latexit><latexit sha1_base64="54T9utMlH0S3Bbo4vShYLLYh0qU=">AAACQnicbVDLShxBFK02D3XMY5Is3RQZBMEg3RJIQAKSbLJUcXwwNTa3a6pnCquqm6rbZoamvy0bv8BdPiAbF4q4dWH12IvxcaDg3HPOpapOkivpMAz/BXMvXr56Pb+w2Fp68/bd+/aHj/suKywXXZ6pzB4m4ISSRnRRohKHuRWgEyUOkpNftX9wKqyTmdnDSS76GoZGppIDeiluH7FdORwhWJv9oWyTbVI2AizHFf1BWWqBl8zJoYZ4fLzBvkzWmsnUE9NFPK5mAjNu1YrbnXA9nII+JVFDOqTBdtw+Z4OMF1oY5Aqc60Vhjv0SLEquRNVihRM58BMYip6nBrRw/XJaQUVXvDKgaWb9MUin6uxGCdq5iU58UgOO3GOvFp/zegWm3/ulNHmBwvD7i9JCUcxo3ScdSCs4qoknwK30b6V8BL449K3XJUSPv/yU7G+sR57vfO1s/WzqWCDL5DNZJRH5RrbIb7JNuoSTv+Q/uSRXwVlwEVwHN/fRuaDZ+UQeILi9A4hdsMA=</latexit><latexit sha1_base64="54T9utMlH0S3Bbo4vShYLLYh0qU=">AAACQnicbVDLShxBFK02D3XMY5Is3RQZBMEg3RJIQAKSbLJUcXwwNTa3a6pnCquqm6rbZoamvy0bv8BdPiAbF4q4dWH12IvxcaDg3HPOpapOkivpMAz/BXMvXr56Pb+w2Fp68/bd+/aHj/suKywXXZ6pzB4m4ISSRnRRohKHuRWgEyUOkpNftX9wKqyTmdnDSS76GoZGppIDeiluH7FdORwhWJv9oWyTbVI2AizHFf1BWWqBl8zJoYZ4fLzBvkzWmsnUE9NFPK5mAjNu1YrbnXA9nII+JVFDOqTBdtw+Z4OMF1oY5Aqc60Vhjv0SLEquRNVihRM58BMYip6nBrRw/XJaQUVXvDKgaWb9MUin6uxGCdq5iU58UgOO3GOvFp/zegWm3/ulNHmBwvD7i9JCUcxo3ScdSCs4qoknwK30b6V8BL449K3XJUSPv/yU7G+sR57vfO1s/WzqWCDL5DNZJRH5RrbIb7JNuoSTv+Q/uSRXwVlwEVwHN/fRuaDZ+UQeILi9A4hdsMA=</latexit><latexit sha1_base64="54T9utMlH0S3Bbo4vShYLLYh0qU=">AAACQnicbVDLShxBFK02D3XMY5Is3RQZBMEg3RJIQAKSbLJUcXwwNTa3a6pnCquqm6rbZoamvy0bv8BdPiAbF4q4dWH12IvxcaDg3HPOpapOkivpMAz/BXMvXr56Pb+w2Fp68/bd+/aHj/suKywXXZ6pzB4m4ISSRnRRohKHuRWgEyUOkpNftX9wKqyTmdnDSS76GoZGppIDeiluH7FdORwhWJv9oWyTbVI2AizHFf1BWWqBl8zJoYZ4fLzBvkzWmsnUE9NFPK5mAjNu1YrbnXA9nII+JVFDOqTBdtw+Z4OMF1oY5Aqc60Vhjv0SLEquRNVihRM58BMYip6nBrRw/XJaQUVXvDKgaWb9MUin6uxGCdq5iU58UgOO3GOvFp/zegWm3/ulNHmBwvD7i9JCUcxo3ScdSCs4qoknwK30b6V8BL449K3XJUSPv/yU7G+sR57vfO1s/WzqWCDL5DNZJRH5RrbIb7JNuoSTv+Q/uSRXwVlwEVwHN/fRuaDZ+UQeILi9A4hdsMA=</latexit>

Published in: Proceedings of 3rd IEEE International Conference on Image Processing.Vol. I, pp. 379-382. Lausanne, Switzerland. 16-19 September 1996.c© IEEE Signal Processing Society, 1996.

NOISE REMOVAL VIA BAYESIAN WAVELET CORING

Eero P. Simoncelli

Computer and Information Science Dept.University of PennsylvaniaPhiladelphia, PA 19104

Edward H. Adelson

Brain and Cognitive Science Dept.Massachusetts Institute of Technology

Cambridge, MA 02139

The classical solution to the noise removal problem isthe Wiener filter, which utilizes the second-order statis-tics of the Fourier decomposition. Subband decomposi-tions of natural images have significantly non-Gaussianhigher-order point statistics; these statistics capture im-age properties that elude Fourier-based techniques. Wedevelop a Bayesian estimator that is a natural exten-sion of the Wiener solution, and that exploits thesehigher-order statistics. The resulting nonlinear esti-mator performs a “coring” operation. We provide asimple model for the subband statistics, and use it todevelop a semi-blind noise-removal algorithm based ona steerable wavelet pyramid.

A common technique for noise reduction is known as“coring”. An image signal is split into two or morebands; the highpass bands are subjected to a thresh-old non-linearity that suppresses low-amplitude valueswhile retaining high-amplitude values. Use of suchtechniques is widespread: for example, most consumerVCR’s use a simple one-dimensional coring technique.Many variants of coring have been developed, includ-ing two-dimensional coring [1], multi-scale oriented cor-ing [2, 3], pyramid coring [4], and multi-band coringwith orthogonal bases [5]. The nonlinear operator is of-ten smoothed to give a “soft” threshold, but the exactchoice of function in these techniques has been ad hoc.Similar techniques, based on the statistical concept of“shrinkage”, have been recently used with wavelet ex-pansions [6].The intuition underlying coring is that images typicallyhave spatial structure, consisting of smooth areas inter-spersed with occasional edges. This notion is evidentstatistically: the pixels in highpass and bandpass sub-bands of images have significantly non-Gaussian proba-bility density functions (pdf’s) that are sharply peakedat zero with broad tails. Specifically, the coefficientof kurtosis κ (fourth moment divided by squared vari-ance) is typically well above the value of 3 that oneexpects for a Gaussian pdf.Field [7] has shown that kurtosis for subbands of nat-ural scenes varies with filter bandwidth, and is maxi-

Research partially supported by an NSF CAREER grant to EPS.

-60 -30 0 30 600

200

400

600

800

1000

-60 -30 0 30 600

200

400

600

800

Figure 1. Histograms of a mid-frequency subbandin an octave-bandwidth wavelet decomposition fortwo different images. Left: The“Einstein” image.Right: A white noise image with uniform pdf.

mal at roughly one octave. Significantly wider or nar-rower bandwidths produce kurtoses near 3 (i.e., Gaus-sian statistics). Several authors have used Laplacianpdf models (with kurtosis 6) for image subband statis-tics (e.g., [8, 9]).Figure 1 contains an example histogram from a singlesubband of a wavelet transform built on the “Einstein”image, for which the sample kurtosis is 9.8. For com-parison, the histogram of the same subband built onuniform white noise is shown. This histogram is nearlyGaussian, with a sample kurtosis of 2.9. Coring relieson the striking dissimilarity between the point statis-tics of these two image types.In the following, we describe a technique for determin-ing the optimal coring function in the Bayesian sense1,and apply it to a steerable wavelet pyramid.

1. BAYESIAN SIGNAL ESTIMATION

Consider a scalar x contaminated with additive noisen: y = x + n. The mean of the posterior distributionprovides an unbiased least-squares estimate of the vari-able x, given measurement y. Bayes’ rule allows us towrite this in terms of the probability densities of thenoise and signal:

x̂(y) =∫

dx Px|y(x|y) x

1An earlier version of this technique is described in [10], abachelor’s thesis supervised by the authors.

y = x+ n

<latexit sha1_base64="jjta6vm/Udp+OzFYfnSI95qsuXA=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSLUS0lE0WPRi8cKpi20oWy2m3bpZjfsbsQS+hu8eFDEqz/Im//GTZuDtj4YeLw3w8y8MOFMG9f9dlZW19Y3Nktb5e2d3b39ysFhS8tUEeoTyaXqhFhTzgT1DTOcdhJFcRxy2g7Ht7nffqRKMykezCShQYyHgkWMYGMlv1l7Oiv3K1W37s6AlolXkCoUaPYrX72BJGlMhSEca9313MQEGVaGEU6n5V6qaYLJGA9p11KBY6qDbHbsFJ1aZYAiqWwJg2bq74kMx1pP4tB2xtiM9KKXi/953dRE10HGRJIaKsh8UZRyZCTKP0cDpigxfGIJJorZWxEZYYWJsfnkIXiLLy+T1nndu6y79xfVxk0RRwmO4QRq4MEVNOAOmuADAQbP8ApvjnBenHfnY9664hQzR/AHzucPgUKN0w==</latexit>

P (x)<latexit sha1_base64="KnIlUwQYq2wbulcBHRlizacqy9E=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBahXkoiih6LXjxWMG2hDWWz3bRLN7thdyOU0N/gxYMiXv1B3vw3btoctPXBwOO9GWbmhQln2rjut1NaW9/Y3CpvV3Z29/YPqodHbS1TRahPJJeqG2JNORPUN8xw2k0UxXHIaSec3OV+54kqzaR4NNOEBjEeCRYxgo2V/FZdnFcG1ZrbcOdAq8QrSA0KtAbVr/5QkjSmwhCOte55bmKCDCvDCKezSj/VNMFkgke0Z6nAMdVBNj92hs6sMkSRVLaEQXP190SGY62ncWg7Y2zGetnLxf+8XmqimyBjIkkNFWSxKEo5MhLln6MhU5QYPrUEE8XsrYiMscLE2HzyELzll1dJ+6LhXTXch8ta87aIowwncAp18OAamnAPLfCBAINneIU3RzgvzrvzsWgtOcXMMfyB8/kDcgaNyQ==</latexit>

P (n)

y = x+ n<latexit sha1_base64="M/nsICnU1MbIVwbdh42x3E1krEg=">AAACEXicbVDLSsNAFJ34rPEVdekmWIS6sCSi6EYounFZwT6wTcNkOmmHTiZhZiItaX7Bjb/ixoUibt2582+ctFlo64ELh3Pu5d57vIgSIS3rW1tYXFpeWS2s6esbm1vbxs5uXYQxR7iGQhrypgcFpoThmiSS4mbEMQw8ihve4DrzGw+YCxKyOzmKsBPAHiM+QVAqyTVK1dLw6LLtc4gSO03uXZHiTnI8nirDNBHpuBOluu4aRatsTWDOEzsnRZCj6hpf7W6I4gAziSgUomVbkXQSyCVBFKd6OxY4gmgAe7ilKIMBFk4y+Sg1D5XSNf2Qq2LSnKi/JxIYCDEKPNUZQNkXs14m/ue1YulfOAlhUSwxQ9NFfkxNGZpZPGaXcIwkHSkCESfqVhP1ocpCqhCzEOzZl+dJ/aRsn5Wt29Ni5SqPowD2wQEoARucgwq4AVVQAwg8gmfwCt60J+1Fe9c+pq0LWj6zB/5A+/wBeeadbw==</latexit>

P (x) =1

Zse�| xs |

p

P (x|y) � P (y|x)P (x)

<latexit sha1_base64="pgbLro3MkdylcMnsSa4wtvqhO/0=">AAACOHicbVBNaxsxENUm/XCdpnXSYy+iJlAoMbsmpb0EQnrprQ7UScDaLLOydi0iaRdpttis92fl0p/RW8klh5aSa39B5Y9Dm/TBwNN7M2jmpaWSDsPwe7Cx+eDho8etJ+2tp9vPnnd2dk9dUVkuhrxQhT1PwQkljRiiRCXOSytAp0qcpZcfFv7ZF2GdLMxnnJUi1pAbmUkO6KWk84lNAOtpc8jA5kxLk0wpUyLDEcss8Ho+25/OL/pN3WdO5hoS4x/0DZ2v7GlTu2Z+UTIr8wnG7XbS6Ya9cAl6n0Rr0iVrDJLONzYueKWFQa7AuVEUlhjXYFFyJZo2q5wogV9CLkaeGtDCxfXy8IbueWVMs8L6MkiX6t8TNWjnZjr1nRpw4u56C/F/3qjC7H1cS1NWKAxffZRVimJBFynSsbSCo5p5AtxKvyvlE/CJoM96EUJ09+T75LTfi972wpOD7tHxOo4WeUlekdckIu/IEflIBmRIOLki1+QH+Rl8DW6CX8HtqnUjWM+8IP8g+P0HrBiuEw==</latexit>

x̂ = argminx

|y � x|2

2�2n

+ |xs|p�

MAP estimate:=

∫dx Py|x(y|x) Px(x) x∫dx Py|x(y|x) Px(x)

=∫

dx Pn(y − x) Px(x) x∫dx Pn(y − x) Px(x)

, (1)

where Pn indicates the probability density function ofthe noise, and Px the prior probability density functionof the signal. The denominator is the pdf of the noisyobservation, computed via convolution of the noise andsignal pdf’s. In order to use this equation to estimatethe original signal value x, we must know both of theseprobability density functions.Consider a few simple examples. First, let the noisehave a zero-mean Gaussian distribution with varianceσ2

n, and let the signal prior be a zero-mean Gaussianwith variance σ2

s . In this case, a well-known closed-form solution exists:

x̂(y) =σ2

s y

σ2s + σ2

n

.

The solution is a simple linear rescaling of the measure-ment. When applied to the coefficients of a Fouriertransform, this estimator corresponds to the Wienerfilter. When applied to subbands of a wavelet trans-form, the solution is an approximation to the Wienerfilter, in which the power spectral density informationis averaged over each of the subbands.Now consider the case in which the noise distributionis Gaussian, but the signal prior is a more sharplypeaked distribution, such as that shown in figure 1. Insuch cases, a closed-form expression for the estimatorin equation (1) may not be available, but a numericalsolution may be used.2. We have computed a numeri-cal approximation of the estimator for the histogramsillustrated figure 1. The estimator is illustrated graph-ically in figure 2. Note that this estimator now corre-sponds to a nonlinear “coring” operation: large ampli-tude values are preserved, and small amplitude valuesare suppressed. This is intuitively sensible: given thesubstantial signal probability mass at x = 0, small val-ues of y are assumed to have arisen from a value ofx = 0. This curve is similar to the soft-thresholdingfunctions that have been previously devised by moread hoc methods; the Bayesian derivation thus providesa principled justification for coring systems.

2. PARAMETERIZED MODEL FORWAVELET COEFFICIENT STATISTICS

The Bayesian estimator discussed above relies on aknowledge of the signal point statistics. In order to uti-lize it, we need a parameterized model for these pdf’ssuch that: 1) the model provides a good fit to the

2One must, in practice, take care to regularize singularitiesresulting from distribution points with very small probability.

-60 -30 0 30 60

-60

-40

-20

0

20

40

60

-60 -30 0 30 60

-60

-40

-20

0

20

40

60

Figure 2. Bayesian estimator (symmetrized) forthe signal and noise histograms shown in figure 1.Superimposed on the plot is a straight line indicat-ing the identity function.

statistics of natural images, and 2) one can estimatethe model parameters from the noisy observation.For our purposes here, we use a two-parameter gener-alized Laplacian distribution, also used by Mallat [11]:

Px(x) ∝ e−|x/s|p . (2)

The distribution is zero-mean and symmetric, and theparameters {s, p} are directly related to the second andfourth moments. Specifically (after consultation withan integral table) one obtains:

σ2 =s2Γ( 3

p )

Γ( 1p )

, κ =Γ( 1

p )Γ( 5p )

Γ2( 3p )

, (3)

where Γ(x) =∫ ∞0 tx−1e−t dt, the well known “gamma”

function. Given the sample variance and kurtosis ofa histogram, we can solve for the two parameters ofour model pdf. Typical values for p are in the range[0.5, 1.0]. This method of model density estimation issimple and direct, but clearly suboptimal. In the cur-rent context, the quality of the estimator should betested by comparing the noise removal results usingthe sample (histogram) statistics, and those using themodel pdf: such a comparison is given in section 4.We are also interested in a more realistic “blind” al-gorithm, in which the parameters are estimated fromnoisy observations. We note that the second and fourthmoments of a generalized Laplacian signal corrupted byadditive Gaussian white noise are:

σ2 = σ2n +

s2Γ( 3p )

Γ( 1p )

, m4 = 3σ4n +

6σ2ns2Γ( 3

p )

Γ( 1p )

+s4Γ( 5

p )

Γ( 1p )

.

Assuming σn is known, the measurements of these twomoments of the noisy data is sufficient to estimate themodel pdf parameters. Results of such an algorithmare given in section 4.

2

<latexit sha1_base64="dw6S4SE9R6K5bh9FoFIidLSnE6s=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0nEoseiF48V7Ae0oWy2m3bpZhN2J2IJ/RFePCji1d/jzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//ci1EbF6wEnC/YgOlQgFo2ildm9EMXua9ssVt+rOQVaJl5MK5Gj0y1+9QczSiCtkkhrT9dwE/YxqFEzyaamXGp5QNqZD3rVU0YgbP5ufOyVnVhmQMNa2FJK5+nsio5ExkyiwnRHFkVn2ZuJ/XjfF8NrPhEpS5IotFoWpJBiT2e9kIDRnKCeWUKaFvZWwEdWUoU2oZEPwll9eJa2LqleruveXlfpNHkcRTuAUzsGDK6jDHTSgCQzG8Ayv8OYkzovz7nwsWgtOPnMMf+B8/gCxjo/N</latexit>

x̂

<latexit sha1_base64="XoMCWGAf2Q0wDLN3HUUhdKkKdl0=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0WPRi8cq1hbaUDbbSbt0swm7G6GE/gMvHhTx6j/y5r9x0+agrQ8GHu/NMDMvSATXxnW/ndLK6tr6RnmzsrW9s7tX3T941HGqGLZYLGLVCahGwSW2DDcCO4lCGgUC28H4JvfbT6g0j+WDmSToR3QoecgZNVa6n1T61Zpbd2cgy8QrSA0KNPvVr94gZmmE0jBBte56bmL8jCrDmcBppZdqTCgb0yF2LZU0Qu1ns0un5MQqAxLGypY0ZKb+nshopPUkCmxnRM1IL3q5+J/XTU145WdcJqlByeaLwlQQE5P8bTLgCpkRE0soU9zeStiIKsqMDScPwVt8eZk8ntW9i7p7d15rXBdxlOEIjuEUPLiEBtxCE1rAIIRneIU3Z+y8OO/Ox7y15BQzh/AHzucPHaONFQ==</latexit>y

(a)

(b)

(c)

(d)

Figure 4. Noise reduction example. (a) Original image (cropped). (b) Image contaminated with additive Gaussianwhite noise (SNR = 9.00dB). (c) Image restored using (semi-blind) Wiener filter (SNR = 11.88dB). (d) Image restoredusing (semi-blind) Bayesian estimator (SNR = 13.82dB).

ful in other applications, such as image compression ortexture synthesis.

6. REFERENCES

[1] J P Rossi. JSMPTE, 87:134–140, 1978.[2] B. E. Bayer and P. G. Powell. A method for the digital

enhancement of unsharp, grainy photographic images.Advances in Computer Vision and Image Processing,2:31–88, 1986.

[3] E P Simoncelli, W T Freeman, E H Adelson, and D JHeeger. Shiftable multi-scale transforms. IEEE TransInformation Theory, 38(2):587–607, March 1992.

[4] C Carlson, E Adelson, and C Anderson. Improvedsystem for coring an image representing signal, 1985.US Patent 4,523,230.

[5] N Ebihara and K Tatsuzawa. Noise reduction system,1979. US Patent 4,163,258.

[6] D L Donoho. Nonlinear wavelet methods for recov-ery of signals, densities, and spectra from indirect andnoisy data. In I. Daubechies, ed, Proc Symp ApplMath, vol 47, Providence, RI, 1993.

[7] D J Field. What is the goal of sensory coding? NeuralComputation, 6:559–601, 1994.

[8] H J Trusell and R P Kruger. Comments on ’nonsta-tionary assumptions for gaussian models in images’.IEEE Trans SMC, SMC-8:579–582, 1978.

[9] A N Netravali and B G Haskell. Digital Pictures.Plenum, New York, 1988.

[10] A. J. Kalb. Noise reduction in video images using cor-ing on QMF pyramids. MIT, EECS Dept Senior The-sis, May 1991.

[11] S G Mallat. A theory for multiresolution signal decom-position: The wavelet representation. IEEE Trans PatAnal Mach Intell, 11:674–693, July 1989.

[12] E P Simoncelli and W T Freeman. The steerablepyramid: A flexible architecture for multi-scale deriva-tive computation. In 2nd Int’l Conf Image Processing,Wash, DC, October 1995.

[13] W T Freeman and E H Adelson. The design and useof steerable filters. IEEE Trans Pat Anal Mach Intell,13(9):891–906, 1991.

4

original image image + noise

Wiener filter wavelet coring

Dynamical models(Kalman filter)

Sparse coding of time-varying images

. . .

t

t

ai(t)

τx

y

x

y

t’

φi(x,y,t−t’)

I(x,y,t)

Learned basis space-time basis functions (200 bfs, 12 x12 x 7)

Sparse coding and reconstruction

0 1 2 3 4 5 6 7

−2

0

2

sparsified

0 1 2 3 4 5 6 7

−2

0

2

convolution

time (sec)

ampl

itude

inference - redwood.berkeley.edu

Documents