{"title": "Discriminative Fields for Modeling Spatial Dependencies in Natural Images", "book": "Advances in Neural Information Processing Systems", "page_first": 1531, "page_last": 1538, "abstract": "", "full_text": "Discriminative Fields for Modeling Spatial\n\nDependencies in Natural Images\n\nSanjiv Kumar and Martial Hebert\n\nThe Robotics Institute\n\nCarnegie Mellon University\n\nPittsburgh, PA 15213\n\nfskumar,hebertg@ri.cmu.edu\n\nAbstract\n\nIn this paper we present Discriminative Random Fields (DRF), a discrim-\ninative framework for the classi\ufb01cation of natural image regions by incor-\nporating neighborhood spatial dependencies in the labels as well as the\nobserved data. The proposed model exploits local discriminative models\nand allows to relax the assumption of conditional independence of the\nobserved data given the labels, commonly used in the Markov Random\nField (MRF) framework. The parameters of the DRF model are learned\nusing penalized maximum pseudo-likelihood method. Furthermore, the\nform of the DRF model allows the MAP inference for binary classi\ufb01ca-\ntion problems using the graph min-cut algorithms. The performance of\nthe model was veri\ufb01ed on the synthetic as well as the real-world images.\nThe DRF model outperforms the MRF model in the experiments.\n\n1\n\nIntroduction\n\nFor the analysis of natural images, it is important to use contextual information in the form\nof spatial dependencies in images. In a probabilistic framework, this leads one to random\n\ufb01eld modeling of the images. In this paper we address the main challenge involving such\nmodeling, i.e. how to model arbitrarily complex dependencies in the observed image data\nas well as the labels in a principled manner.\n\nIn the literature, Markov Random Field (MRF) is a commonly used model to incorporate\ncontextual information [1]. MRFs are generally used in a probabilistic generative frame-\nwork that models the joint probability of the observed data and the corresponding labels.\nIn other words, let y be the observed data from an input image, where y = fyigi2S, yi\nis the data from ith site, and S is the set of sites. Let the corresponding labels at the im-\nage sites be given by x = fxigi2S. In the MRF framework, the posterior over the labels\ngiven the data is expressed using the Bayes\u2019 rule as, p(xjy) / p(x; y) = p(x)p(yjx)\nwhere the prior over labels, p(x) is modeled as a MRF. For computational tractability, the\nobservation or likelihood model, p(yjx) is usually assumed to have a factorized form, i.e.\nsumption is too restrictive for the analysis of natural images. For example, consider a class\nthat contains man-made structures (e.g. buildings). The data belonging to such a class is\nhighly dependent on its neighbors since the lines or edges at spatially adjoining sites follow\n\np(yjx) = Qi2S p(yijxi)[1][2]. However, as noted by several researchers [3][4], this as-\n\n\fsome underlying organization rules rather than being random (See Fig. 2). This is also true\nfor a large number of texture classes that are made of structured patterns.\n\nSome efforts have been made in the past to model the dependencies in the data [3][4], but\nthey make factored approximations of the actual likelihood for tractability. In addition,\nsimplistic forms of the factors preclude capturing stronger relationships in the observations\nin the form of arbitrarily complex features that might be desired to discriminate between\ndifferent classes. Now considering a different point of view, for classi\ufb01cation purposes, we\nare interested in estimating the posterior over labels given the observations, i.e., p(xjy).\nIn a generative framework, one expends efforts to model the joint distribution p(x; y),\nwhich involves implicit modeling of the observations. In a discriminative framework, one\nmodels the distribution p(xjy) directly. As noted in [2], a potential advantage of using the\ndiscriminative approach is that the true underlying generative model may be quite complex\neven though the class posterior is simple. This means that the generative approach may\nspend a lot of resources on modeling the generative models which are not particularly\nrelevant to the task of inferring the class labels. Moreover, learning the class density models\nmay become even harder when the training data is limited [5].\n\nIn this work we present a Discriminative Random Field (DRF) model based on the con-\ncept of Conditional Random Field (CRF) proposed by Lafferty et al. [6] in the context of\nsegmentation and labeling of 1-D text sequences. The CRFs directly model the posterior\ndistribution p(xjy) as a Gibbs \ufb01eld. This approach allows one to capture arbitrary de-\npendencies between the observations without resorting to any model approximations. Our\nmodel further enhances the CRFs by proposing the use of local discriminative models to\ncapture the class associations at individual sites as well as the interactions in the neigh-\nboring sites on 2-D grid lattices. The proposed model uses local discriminative models to\nachieve the site classi\ufb01cation while permitting interactions in both the observed data and\nthe label \ufb01eld in a principled manner. The research presented in this paper alleviates several\nproblems with the previous version of the DRFs described in [7].\n\n2 Discriminative Random Field\n\nWe \ufb01rst restate in our notations the de\ufb01nition of the Conditional Random Fields as given\nby Lafferty et al. [6]. In this work we will be concerned with binary classi\ufb01cation, i.e.\nxi 2 f(cid:0)1; 1g. Let the observed data at site i, yi 2 <c.\nCRF De\ufb01nition: Let G = (S; E) be a graph such that x is indexed by the vertices of G.\nThen (x; y) is said to be a conditional random \ufb01eld if, when conditioned on y, the ran-\ndom variables xi obey the Markov property with respect to the graph: p(xijy; xS(cid:0)fig) =\np(xijy; xNi ), where S(cid:0)fig is the set of all nodes in G except the node i, Ni is the set of\nneighbors of the node i in G, and x(cid:10) represents the set of labels at the nodes in set (cid:10).\nThus CRF is a random \ufb01eld globally conditioned on the observations y. The condition of\npositivity requiring p(xjy) > 0 8 x has been assumed implicitly. Now, using the Hammer-\nsley Clifford theorem [1] and assuming only up to pairwise clique potentials to be nonzero,\nthe joint distribution over the labels x given the observations y can be written as,\n\np(xjy) =\n\n1\nZ\n\n(1)\n\nexp0@Xi2S\n\nAi(xi; y)+Xi2S Xj2Ni\n\nIij(xi; xj; y)1A\n\nwhere Z is a normalizing constant known as the partition function, and -Ai and -Iij are the\nunary and pairwise potentials respectively. With a slight abuse of notations, in the rest of\nthe paper we will call Ai as association potential and Iij as interaction potential. Note that\nboth the terms explicitly depend on all the observations y. In the DRFs, the association\npotential is seen as a local decision term which decides the association of a given site to a\ncertain class ignoring its neighbors. The interaction potential is seen as a data dependent\n\n\fsmoothing function. For simplicity, in the rest of the paper we assume the random \ufb01eld\ngiven in (1) to be homogeneous and isotropic, i.e. the functional forms of Ai and Iij are\nindependent of the locations i and j. Henceforth we will leave the subscripts and simply\nuse the notations A and I. Note that the assumption of isotropy can be easily relaxed at the\ncost of a few additional parameters.\n\n2.1 Association potential\n\nIn the DRF framework, A(xi; y) is modeled using a local discriminative model that outputs\nthe association of the site i with class xi. Generalized Linear Models (GLM) are used\nextensively in statistics to model the class posteriors given the observations [8]. For each\nsite i, let f i(y) be a function that maps the observations y on a feature vector such that\nf i : y ! <l. Using a logistic function as the link, the local class posterior can be modeled\nas,\n\nP (xi = 1jy) =\n\n1\n\n1+e(cid:0)(w0+wT\n\n1 f i(y))\n\n= (cid:27)(w0 + wT\n\n1 f i(y))\n\n(2)\n\nwhere w = fw0; w1g are the model parameters. To extend the logistic model to induce a\nnonlinear decision boundary in the feature space, a transformed feature vector at each site\ni is de\ufb01ned as, hi(y) = [1; (cid:30)1(f i(y)); : : : ; (cid:30)R(f i(y))]T where (cid:30)k(:) are arbitrary non-\nlinear functions. The \ufb01rst element of the transformed vector is kept as 1 to accommodate\nthe bias parameter w0. Further, since xi 2 f(cid:0)1; 1g, the probability in (2) can be compactly\nexpressed as P (xijy) = (cid:27)(xiwT hi(y)). Finally, the association potential is de\ufb01ned as,\n(3)\nThis transformation makes sure that the DRF yields standard logistic classi\ufb01er if the inter-\naction potential in (1) is set to zero. Note that the transformed feature vector at each site\ni, i.e. hi(y) is a function of whole set of observations y. On the contrary, the assumption\nof conditional independence of the data in the MRF framework allows one to use the data\nonly from a particular site, i.e. yi to get the log-likelihood, which acts as the association\npotential.\n\nA(xi; y) = log((cid:27)(xiwT hi(y))\n\nAs a related work, in the context of tree-structured belief networks, Feng et al. [2] used\nthe scaled likelihoods to approximate the actual likelihoods at each site required by the\ngenerative formulation. These scaled likelihoods were obtained by scaling the local class\nposteriors learned using a neural network. On the contrary, in the DRF model, the local\nclass posterior is an integral part of the full conditional model in (1). Also, unlike [2], the\nparameters of the association and interaction potential are learned simultaneously in the\nDRF framework.\n\n2.2\n\nInteraction potential\n\nTo model the interaction potential I, we \ufb01rst analyze the interaction potential commonly\nused in the MRF framework. Note that the MRF framework does not permit the use of data\nin the interaction potential. For a homogeneous and isotropic Ising model, the interaction\npotential is given as I = (cid:12)xixj, which penalizes every dissimilar pair of labels by the cost\n(cid:12) [1]. This form of interaction prefers piecewise constant smoothing without explicitly\nconsidering discontinuities in the data. In the DRF formulation, the interaction potential is\na function of all the observations y. We would like to have similar labels at a pair of sites\nfor which the observed data supports such a hypothesis. In other words, we are interested\nin learning a pairwise discriminative model as the interaction potential.\nFor a pair of sites (i; j), let (cid:22)ij( i(y);  j(y)) be a new feature vector such that (cid:22)ij :<(cid:13) (cid:2)\n<(cid:13) ! <q, where  k : y ! <(cid:13). Denoting this feature vector as (cid:22)ij(y) for simpli\ufb01cation,\nthe interaction potential is modeled as,\n(4)\n\nI(xi; xj; y) = xixjvT (cid:22)ij(y)\n\n\fwhere v are the model parameters. Note that the \ufb01rst component of (cid:22)ij(y) is \ufb01xed to be\n1 to accommodate the bias parameter. This form of interaction potential is much simpler\nthan the one proposed in [7], and makes the parameter learning a convex problem. There\nare two interesting properties of the interaction potential given in (4). First, if the associ-\nation potential at each site and the interaction potentials of all the pairwise cliques except\nthe pair (i; j) are set to zero in (1), the DRF acts as a logistic classi\ufb01er which yields the\nprobability of the site pair to have the same labels given the observed data. Second, the pro-\nposed interaction potential is a generalization of the Ising model. The original Ising form\nis recovered if all the components of vector v other than the bias parameter are set to zero\nin (4). Thus, the form in (4) acts as a data-dependent discontinuity adaptive model that will\nmoderate smoothing when the data from the two sites is \u2019different\u2019. The data-dependent\nsmoothing can especially be useful to absorb the errors in modeling the association poten-\ntial. Anisotropy can be easily included in the DRF model by parametrizing the interaction\npotentials of different directional pairwise cliques with different sets of parameters v.\n\n3 Parameter learning and inference\nLet (cid:18) be the set of DRF parameters where (cid:18) = fw; vg. The form of the DRF model\nresembles the posterior of the MRF framework assuming conditionally independent data.\nHowever, in the MRF framework, the parameters of the class generative models, p(y ijxi)\nand the parameters of the prior random \ufb01eld on labels, p(x) are generally assumed to be\nindependent and learned separately [1]. In contrast, we make no such assumption and learn\nall the parameters of the DRF simultaneously.\n\nThe maximum likelihood approach to learn the DRF parameters involves evaluation of the\npartition function Z which is in general a NP-hard problem. One could use either sam-\npling techniques or resort to some approximations e.g. pseudo-likelihood to estimate the\nparameters. In this work we used the pseudo-likelihood formulation due to its simplicity\nand consistency of the estimates for the large lattice limit [1]. In the pseudo-likelihood\n\napproach, a factored approximation is used such that, P (xjy; (cid:18)) (cid:25)Qi2S P (xijxNi ; y; (cid:18)).\n\nHowever, for the Ising model in MRFs, pseudo-likelihood tends to overestimate the inter-\naction parameter (cid:12), causing the MAP estimates of the \ufb01eld to be very poor solutions [9].\nOur experiments in the previous work [7] and Section 4 of this paper verify these obser-\nvations for the interaction parameters in DRFs too. To alleviate this problem, we take a\nBayesian approach to get the maximum a posteriori estimates of the parameters. Similar\nto the concept of weight decay in neural learning literature, we assume a Gaussian prior\nover the interaction parameters v such that p(vj(cid:28) ) = N (v; 0; (cid:28) 2I) where I is the identity\nmatrix. Using a prior over parameters w that leads to weight decay or shrinkage might\nalso be bene\ufb01cial but we leave that for future exploration. The prior over parameters w is\nassumed to be uniform. Thus, given M independent training images,\n\nb(cid:18) = arg max\n\n(cid:18)\n\nwhere\n\n8<:\nMXm=1Xi2S\nlog (cid:27)(xiwT hi(y))+Xj2Ni\nexp8<:\nzi = Xxi2f(cid:0)1;1g\n\nxixjvT(cid:22)ij(y)(cid:0)log zi9=;(cid:0)\nxixjvT (cid:22)ij(y)9=;\n\nlog (cid:27)(xiwT hi(y)) + Xj2Ni\n\n1\n2(cid:28) 2 vT v (5)\n\nIf (cid:28) is given, the penalized log pseudo-likelihood in (5) is convex with respect to the model\nparameters and can be easily maximized using gradient descent.\n\nAs a related work regarding the estimation of (cid:28) , Mackay [10] has suggested the use of\ntype II marginal likelihood. But in the DRF formulation, integrating the parameters v\nis a hard problem. Another choice is to integrate out (cid:28) by choosing a non-informative\n\n\fhyperprior on (cid:28) as in [11] [12]. However our experiments showed that these methods\ndo not yield good estimates of the parameters because of the use of pseudo-likelihood in\nour framework. In the present work we choose (cid:28) by cross-validation. Alternative ways\nof parameter estimation include the use of contrastive divergence [13] and saddle point\napproximations resembling perceptron learning rules [14]. We are currently exploring these\npossibilities.\n\nThe problem of inference is to \ufb01nd the optimal label con\ufb01guration x given an image y,\nwhere optimality is de\ufb01ned with respect to a cost function. In the current work we use the\nMAP estimate as the solution to the inference problem. While using the Ising MRF model\nfor the binary classi\ufb01cation problems, exact MAP solution can be computed using min-\ncut/max-\ufb02ow algorithms provided (cid:12) (cid:21) 0 [9][15]. For the DRF model, the MAP estimates\ncan be obtained using the same algorithms. However, since these algorithms do not allow\nnegative interaction between the sites, the data-dependent smoothing for each clique is set\nto be vT(cid:22)ij(y) = maxf0; vT(cid:22)ij(y)g, yielding an approximate MAP estimate. This is\nequivalent to switching the smoothing off at the image discontinuities.\n\n4 Experiments and discussion\n\nFor comparison, a MRF framework was also learned assuming a conditionally indepen-\ndent likelihood and a homogeneous and isotropic Ising interaction model. So, the MRF\nposterior is p(xjy) = Z(cid:0)1\nis the interaction parameter and si(yi) is a single-site feature vector at ith site such that\nsi : yi ! <d. Note that si(yi) does not take into account in\ufb02uence of the data in the\nneighborhood of ith site. A \ufb01rst order neighborhood (4 nearest neighbors) was used for\nlabel interaction in all the experiments.\n\nm exp(cid:16)Pi2S log p(si(yi)jxi) +Pi2SPj2Ni\n\n(cid:12)xixj(cid:17) where (cid:12)\n\n4.1 Synthetic images\nThe aim of these experiments was to obtain correct labels from corrupted binary images.\nFour base images, 64 (cid:2) 64 pixels each, were used in the experiments (top row in Fig. 1).\nWe compare the DRF and the MRF results for two different noise models. For each noise\nmodel, 50 images were generated from each base image. Each pixel was considered as an\nimage site and the feature vector si(yi) was simply chosen to be a scalar representing the\nintensity at ith site. In experiments with the synthetic data, no neighborhood data interac-\ntion was used for the DRFs (i.e. f i(y) = si(yi)) to observe the gains only due to the use of\ndiscriminative models in the association and interaction potentials. A linear discriminant\nwas implemented in the association potential such that hi(y) = [1; f i(y)]T . The pairwise\ndata vector (cid:22)ij(y) was obtained by taking the absolute difference of si(yi) and sj(yj).\nFor the MRF model, each class-conditional density, p(si(yi)jxi), was modeled as a Gaus-\nsian. The noisy data from the left most base image in Fig.1 was used for training while 150\nnoisy images from the rest of the three base images were used for testing.\n\nThree experiments were conducted for each noise model. (i) The interaction parameters for\nthe DRF (v) as well as for the MRF ((cid:12)) were set to zero. This reduces the DRF model to a\nlogistic classi\ufb01er and MRF to a maximum likelihood (ML) classi\ufb01er. (ii) The parameters of\nthe DRF, i.e. [w; v], and the MRF, i.e. (cid:12), were learned using pseudo-likelihood approach\nwithout any penalty, i.e. (cid:28) = 1. (iii) Finally, the DRF parameters were learned using\npenalized pseudo-likelihood and the best (cid:12) for the MRF was chosen from cross-validation.\nThe MAP estimates of the labels were obtained using graph-cuts for both the models.\n\nUnder the \ufb01rst noise model, each image pixel was corrupted with independent Gaussian\nnoise of standard deviation 0:3. For the DRF parameter learning, (cid:28) was chosen to be\n0:01. The pixelwise classi\ufb01cation error for this noise model is given in the top row of\nTable 1. Since the form of noise is the same as the likelihood model in the MRF, MRF is\n\n\fTable 1: Pixelwise classi\ufb01cation errors (%) on 150 synthetic test images. For the Gaussian\nnoise MRF and DRF give similar error while for \u2019bimodal\u2019 noise, DRF performs better.\nNote that only label interaction (i.e. no data interaction) was used for these tests (see text).\n\nNoise ML\n15:62\n24:00\n\nGaussian\nBimodal\n\nLogistic MRF (PL) DRF (PL) MRF DRF\n15:78\n2:30\n6:21\n29:86\n\n29:49\n29:49\n\n13:18\n22:70\n\n2:35\n7:00\n\nFigure 1: Results on synthetic images. From top, \ufb01rst row:original images, second row:\nimages corrupted with \u2019bimodal\u2019 noise, third row: MRF results, fourth row: DRF results.\n\nexpected to give good results. The DRF model does marginally better than MRF even for\nthis case. Note that the DRF and MRF results are worse when the parameters were learned\nwithout penalizing the pseudo-likelihood (shown in Table 1 with suf\ufb01x (PL)). The MAP\ninference yields oversmoothed images for these parameters. The DRF model is affected\nmore because all the parameters in DRFs are learned simultaneously unlike MRFs.\n\nIn the second noise model each pixel was corrupted with independent mixture of Gaus-\nsian noise. For each class, a mixture of two Gaussians with equal mixing weights was\nused yielding a \u2019bimodal\u2019 class noise. The mixture model parameters (mean, std) for the\ntwo classes were chosen to be [(0:08; 0:03); (0:46; 0:03)], and [(0:55; 0:02); (0:42; 0:10)]\ninspired by [5]. The classi\ufb01cation results are shown in the bottom row of Table 1. An\ninteresting point to note is that DRF yields lower error than MRF even when the logistic\nclassi\ufb01er has higher error than the ML classi\ufb01er on the test data. For a typical noisy version\nof the four base images, the performance of different techniques in compared in Fig. 1.\n\n\fTable 2: Detection Rates (DR) and False Positives (FP) for the test set containing 129\nimages (49; 536 sites). FP for logistic classi\ufb01er were kept to be the same as for DRF for\nDR comparison. Superscript 0(cid:0)0 indicates no neighborhood data interaction was used.\n\nDR (%)\nFP (per image)\n\n4.2 Real-World images\n\nMRF\n58:35\n2:44\n\nLogistic(cid:0) DRF(cid:0)\n61:79\n2:28\n\n47:50\n2:28\n\nLogistic DRF\n60:80\n72:54\n1:76\n1:76\n\nThe proposed DRF model was applied to the task of detecting man-made structures in\nnatural scenes. The aim was to label each image site as structured or nonstructured. The\ntraining and the test set contained 108 and 129 images respectively, each of size 256(cid:2)384\npixels, from the Corel image database. Each nonoverlapping 16(cid:2)16 pixels block is called\nan image site. For each image site i, a 5-dim single-site feature vector si(yi) and a 14-dim\nmultiscale feature vector f i(y) is computed using orientation and magnitude based features\nas described in [16]. Note that f i(y) incorporates data interaction from neighboring sites.\nFor the association potentials, a transformed feature vector hi(y) was computed at each site\ni using quadratic transforms of vector f i(y). The pairwise data vector (cid:22)ij(y) was obtained\nby concatenating the two vectors f i(y) and f j(y). For the DRF parameter learning, (cid:28) was\nchosen to be 0:001. For the MRF, each class conditional density was modeled as a mixture\nof \ufb01ve Gaussians. Use of a single Gaussian for each class yielded very poor results.\n\nFor two typical images from the test set, the detection results for the MRF and the DRF\nmodels are given in Fig. 2. The blocks detected as structured have been shown enclosed\nwithin an arti\ufb01cial boundary. The DRF results show higher detections with lower false\npositives. For a quantitative evaluation, we compared the detection rates and the number\nof false positives per image for different techniques. For the comparison of detection rates,\nin all the experiments, the decision threshold of the logistic classi\ufb01er was \ufb01xed such that it\nyields the same false positive rate as the DRF. The \ufb01rst set of experiments was conducted\nusing the single-site features for all the three methods. Thus, no neighborhood data in-\nteraction was used for both the logistic classi\ufb01er and the DRF, i.e. f i(y) = si(yi). The\ncomparative results for the three methods are given in Table 2 under \u2019MRF\u2019, \u2019Logistic(cid:0)\u2019\nand \u2019DRF(cid:0)\u2019. The detection rates of the MRF and the DRF are higher than the logistic clas-\nsi\ufb01er due to the label interaction. However, higher detection rate and lower false positives\nfor the DRF in comparison to the MRF indicate the gains due to the use of discriminative\nmodels in the association and interaction potentials in the DRF. In the next experiment,\nto take advantage of the power of the DRF framework, data interaction was allowed for\nboth the logistic classi\ufb01er as well as the DRF (\u2019Logistic\u2019 and \u2019DRF\u2019 in Table 2). The DRF\ndetection rate increases substantially and the false positives decrease further indicating the\nimportance of allowing the data interaction in addition to the label interaction.\n\n5 Conclusion and future work\n\nWe have presented discriminative random \ufb01elds which provide a principled approach for\ncombining local discriminative classi\ufb01ers that allow the use of arbitrary overlapping fea-\ntures, with adaptive data-dependent smoothing over the label \ufb01eld. We are currently explor-\ning alternative ways of parameter learning using contrastive divergence and saddle point\napproximations. One of the further aspects of the DRF model is the use of general kernel\nmappings to increase the classi\ufb01cation accuracy. However, one will need some method to\ninduce sparseness to avoid over\ufb01tting [12]. In addition, we intend to extend the model to\naccommodate multiclass classi\ufb01cation problems.\n\nAcknowledgments\nOur thanks to John Lafferty and Jonas August for immensely helpful discussions.\n\n\fFigure 2: Example structure detection results. Left column: MRF results. Right column:\nDRF results. DRF has higher detection rate with lower false positives.\n\nReferences\n\n[1] S. Z. Li. Markov Random Field Modeling in Image Analysis. Springer-Verlag, Tokyo, 2001.\n[2] X. Feng, C. K. I. Williams, and S. N. Felderhof. Combining belief networks and neural networks\n\nfor scene segmentation. IEEE Trans. Pattern Anal. Machine Intelli., 24(4):467\u2013483, 2002.\n\n[3] H. Cheng and C. A. Bouman. Multiscale bayesian segmentation using a trainable context model.\n\nIEEE Trans. on Image Processing, 10(4):511\u2013525, 2001.\n\n[4] R. Wilson and C. T. Li. A class of discrete multiresolution random \ufb01elds and its application to\n\nimage segmentation. IEEE Trans. on Pattern Anal. and Machine Intelli., 25(1):42\u201356, 2003.\n\n[5] Y. D. Rubinstein and T. Hastie. Discriminative vs informative learning. In Proc. Third Int. Conf.\n\non Knowledge Discovery and Data Mining, pages 49\u201353, 1997.\n\n[6] J. Lafferty, A. McCallum, and F. Pereira. Conditional random \ufb01elds: Probabilistic models for\n\nsegmenting and labeling sequence data. In Proc. Int. Conf. on Machine Learning, 2001.\n\n[7] S. Kumar and M. Hebert. Discriminative random \ufb01elds: A discriminative framework for con-\ntextual interaction in classi\ufb01cation. IEEE Int. Conf. on Computer Vision, 2:1150\u20131157, 2003.\n[8] P. McCullagh and J. A. Nelder. Generalised Linear Models. Chapman and Hall, London, 1987.\n[9] D. M. Greig, B. T. Porteous, and A. H. Seheult. Exact maximum a posteriori estimation for\n\nbinary images. Journal of Royal Statis. Soc., 51(2):271\u2013279, 1989.\n\n[10] D. Mackay. Bayesian non-linear modelling for the 1993 energy prediction competition.\n\nMaximum Entropy and Bayesian Methods, pages 221\u2013234, 1996.\n\nIn\n\n[11] P. Williams. Bayesian regularization and pruning using a laplacian prior. Neural Computation,\n\n7:117\u2013143, 1995.\n\n[12] M. A. T. Figueiredo. Adaptive sparseness using jeffreys prior. Advances in Neural Information\n\nProcessing Systems (NIPS), 2001.\n\n[13] G. E. Hinton. Training product of experts by minimizing contrastive divergence. Neural Com-\n\nputation, 14:1771\u20131800, 2002.\n\n[14] M. Collins. Discriminative training methods for hidden markov models: Theory and exper-\nIn Proc. Conference on Empirical Methods in Natural\n\niments with perceptron algorithms.\nLanguage Processing (EMNLP), 2002.\n\n[15] V. Kolmogorov and R. Zabih. What energy functions can be minimized via graph cuts. In Proc.\n\nEuropean Conf. on Computer Vision, 3:65\u201381, 2002.\n\n[16] S. Kumar and M. Hebert. Man-made structure detection in natural images using a causal mul-\ntiscale random \ufb01eld. In Proc. IEEE Int. Conf. on Comp. Vision and Pattern Recog., June 2003.\n\n\f", "award": [], "sourceid": 2352, "authors": [{"given_name": "Sanjiv", "family_name": "Kumar", "institution": null}, {"given_name": "Martial", "family_name": "Hebert", "institution": null}]}