A new sampling-based Bayesian approach for fractionally integrated autoregressive moving average (ARFIMA) processes is presented. A particular type of ARMA process is used as an approximation for the ARFIMA in a Metropolis-Hastings algorithm, and then importance sampling is used to adjust for the approximation error. This algorithm is relatively time-efficient because of fast convergence in the sampling procedures and fewer computations than competitors. Its frequentist properties are investigated through a simulation study. The performance of the posterior means is quite comparable to that of the maximum likelihood estimators for small samples, but the algorithm can be extended easily to a variety of related processes, including ARFIMA plus short-memory noise. The methodology is illustrated using the Nile River data.